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Preface 


In  the  last  decade  it  has  become  apparent  that  paper- 
based  technical  orders  (TO)  are  obsolete  for  a  modern 
defense  force  in  terms  of  efficiency  and  cost.  In  the  near 
future,  possibly  10  years.  Air  Force  technicians  will  use 
automated  TOs  to  access,  display,  and  store  information  to 
operate  and  maintain  Air  Force  equipment  and  resources.  The 
Armstrong  Laboratory  at  Wright-Patterson  AFB,  OH  has  been 
researching  and  experimenting  with  electronic  medium  devices 
for  over  12  years  and  they  have  developed  a  ruggedized 
computer  for  use  on  the  flight  line.  It  was  our  distinct 
pleasure  to  assist  the  laboratory  in  this  endeavor  by 
testing  two  types  of  video  access  devices  -  a  joystick  and  a 
four-button  cluster  cursor  key. 

For  this  experiment  we  tested  a  joystick  device  against 
a  four-button  cluster  cursor  key  to  determine  which  access 
method  interfaced  with  a  computer  the  fastest,  and  which 
device  provided  the  most  user  satisfaction.  The  results  of 
our  research  are  presented  in  Chapter  IV  and  in  appendices  H 
and  I. 

Completion  of  this  thesis  was  one  of  the  most  rewarding 
experiences  at  AFIT  because  it  allowed  us  to  apply  the 
knowledge  and  skill  we  gained  throughout  the  academic  year. 
With  that  in  mind  we  take  this  opportunity  to  thank  all  our 
professors  and  advisors.  Although  the  homework  was  over¬ 
bearing,  the  classroom  lectures  were  boring,  and  the  tests 
were  unrealistic,  the  maturing  process  that  "now"  allows  us 
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to  think  critically  was  worth  it.  Even  we  were  surprised  at 
what  we  truly  understood  and  were  able  to  apply  when  it  came 
time  to  accomplish  this  thesis.  Thank  you!!!! 

We  especially  thank  our  thesis  advisors  Art  Munguia, 
and  Barbara  Masquelier.  They  provided  us  a  wide  latitude  in 
which  to  perform  our  thesis  research,  but  were  always  there 
to  advise  us  and  to  set  us  back  on  course.  We  began  this 
experiment  with  a  pretty  in-depth  knowledge  of  technical 
orders,  or  so  we  thought,  however.  Art  soon  showed  us  the 
error  of  our  thinking  and  skillfully  expanded  our  knowledge 
of  technical  orders  by  a  hundred-fold.  Thanks  Art  for  your 
knowledge,  time  and  patience. 

Barbara  was  instrumental  in  helping  us  to  approach  this 
experiment  in  a  critical  manner  by  forcing  us  to  carefully 
plan  every  detail  and  just  when  we  thought  we  had  the 
perfect  plan,  she  would  offer  us  another  insight.  After 
many  attempts  we  did  succeed  in  developing  a  near  flawless 
plan  and  everything  progressed  smoothly.  Thank  you  Barbara 
for  introducing  us  to  the  scientific  aspect  of  the  thesis 
and  for  answering  all  those  Saturday  telephone  calls.  And 
please  thank  everyone  at  the  lab  who  assisted  us  in  this 
experiment.  Especially  those  we  inconvenienced  by  stealing 
their  computer. 

We  also  offer  our  sincere  thanks  to  our  families  and 
friends  who  endured  these  15  months  with  us.  We  found  that 
late  night  study  sessions  and  last  minute  paper  changes  not 
only  affected  us  but  also  affected  the  whole  family. 
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Everyone  in  the  house  lost  sleep  when  we  were  working  in 
what  was  affectionately  called  "daddies  dungeon".  We  look 
forward  to  sharing  more  of  our  lives  with  you  and  promise  to 
leave  the  companionship  of  our  computers  for  a  night  on  the 
town.  How  about  dinner? 

Gerald  Streff  Robert  Gundel 
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Abstract 

The  purpose  of  this  thesis  was  to  evaluate  the  use  of 
four-cursor  buttons  versus  a  joystick  device  to  present 
technical  order  information  on  a  portable  maintenance  aid 
computer  for  aircraft  technicians  in  a  maintenance 
environment.  Studies  to  date  have  verified  the  superiority 
of  computer-presented  technical  orders  when  compared  to 
current  paper-based  systems;  this  research  investigated  the 
effectiveness  of  computer-based  systems  when  technicians 
used  »  ioystick  or  four-cursor  buttons  to  display  desired 
technical  information.  A  comparison  was  made  as  to 
technician  performance  using  a  portable  maintenance  aid  at 
the  4950th  Test  Wing,  WPAFB  OH.  A  total  of  thirty-two 
maintenance  technicians  performed  two  simulated  tasks  using 
a  F-18  demonstration  program.  The  technicians  accessed  and 
displayed  technical  information  using  the  joystick  device 
and  the  four  cursor-keys.  Results  indicated  no  significant 
difference  in  the  performance  of  maintenance  technicians 
when  using  either  access  device  in  a  controlled  environment. 
The  major  conclusion  was  that  either  access  device  may  be  a 
viable  answer  for  use  in  a  flight  line  environment.  The 
chief  recommendation  is  that  further  studies  be  performed 
using  a  different  joystick  device  and  computer  software  that 
supports  all  eight  joystick  functions. 
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COMPARISON  OF  FOUR-CURSOR  BUTTONS  VS  JOYSTICK  TO 


ACCESS  COMPUTERIZED  TECHNICAL  INFORMATION  FROM 
AN  INTEGRATED  MAINTENANCE  INFORMATION  SYSTEM 

1.  Introduction 

Accurate  dissemination  of  information  is  an  essential 
part  of  the  United  States  Air  Force  whether  it  is  in  the 
form  of  combat  orders,  equipment  operating  instructions, 
safety  procedures,  or  technical  orders  (TO) .  Every  day.  Air 
Force  personnel  use  written  communications  to  transfer 
information  from  one  source  to  another.  If  information  is 
inadequately  communicated,  the  results  can  be  life 
threatening  -  improperly  conveyed  safety  procedures  could 
result  in  death  or  equipment  damage.  It  is  extremely 
important  for  all  information  to  be  factual,  timely,  and 
concise  if  it  is  to  serve  the  needs  of  the  user  or  reader. 
For  all  Air  Force  personnel  -  aircraft  pilots,  x-ray  machine 
operators,  maintenance  technicians,  flight  engineers,  car 
mechanics,  aerospace  engineers  -  their  TOs  must  be  accurate 
and  convey  information  effectively  for  them  to  perform  their 
jobs  correctly  and  safely. 

Although  the  paper  based  TO  system  has  adequately 
served  the  needs  of  the  Air  Force  since  1940,  it  can  no 
longer  support  the  volume  of  information  created  by 
technological  advances  and  sophisticated  wei;pon  system 
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acquisitions.  "TOs  are  too  costly  to  update  and  maintain, 
too  cumbersome  to  use,  and  too  difficult  to  secure”  (1:5). 

As  outlined  in  Chapter  Two  of  TO  00-5-1,  the  Air  Force 
Technical  Order  System,  there  are  five  basic  categories  of 
TOs  that  provide  technical  information  to  operate,  maintain, 
and  modify  Air  Force  equipment.  TO  use  is  mandatory  for  all 
Air  Force  personnel  while  operating  or  maintaining  Air  Force 
or  Air  Force  contracted  equipment  (2:1). 

The  largest  TO  category  is  the  Operations  and 
Maintenance  Technical  Orders.  These  TOs  describe 
installation,  operation,  maintenance  and  handling  of  Air 
Force  equipment  and  material.  This  series  is  further  broken 
down  into  seven  separate  manuals  each  having  their  own  TO 
control  number.  See  figure  1  on  page  4  for  a  breakdown  of 
the  different  categories  of  TOs  (3:2-1). 

The  second  major  category  is  the  Methods  and  Procedures 
Technical  Order.  These  TOs  establish  policies  and  prescribe 
procedures  for  the  care  and  use  of  Air  Force  equipment. 
Within  this  category  there  are  six  separate  sub-sections 
including  job  guides,  air  crew  flight  manual  publications, 
etc.  Each  sub-section  has  its  own  TO  control  number  and 
areas  of  interest  (3:2-2). 

Abbreviated  Technical  Orders  simplify  equipment 
inspection  routines  by  providing  repair  technicians  with 
condensed  forms  of  larger  system  TOs.  Abbreviated  TOs 
contain  checklists,  step-by-step  equipment  inspection 
routines,  inspection  charts,  and  lubrication  charts.  Within 


2 


this  series,  there  are  three  sub-sections  that  contain 
specific  information  controlled  by  separate  TO  numbers. 

These  TOs  are  used  extensively  by  maintenance  personnel 
(3:2-2) . 

The  fourth  major  category  of  TOs  is  the  Time  Compliance 
Technical  Order  (TCTO) .  This  series  controls  equipment 
modifications  by  authorizing  and  standardizing  the 
modification  process.  TCTOs  provide  detailed  modification 
instructions  to  technicians  so  that  all  modified  equipment 
meets  established  operating  specifications.  There  are  five 
sub-classes  within  this  TO  category.  As  seen  in  the  other 
TO  categories,  these  sub-classes  are  controlled  by  their  own 
TO  number  (3:2-3). 

The  fifth  type  of  TO  is  the  Index  Type  Technical  Order. 
These  TOs  provide  Air  Force  managers,  operators,  and 
technicians  with  the  means  to  track  and  identify  the  active 
TOs  currently  authorized  for  use  on  Air  Force  equipment. 
Index  Type  Technical  Orders  are  broken  down  into  four  major 
categories.  As  previously  noted  in  the  other  categories, 
each  sub-category  has  its  own  TO  number  (3:2-2). 
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Air  Force 
TO  System 


Figure  1.  Technical  Order  Publications  Breakout 


It  soon  becomes  apparent  that  a  modern  weapon  system 
can  easily  generate  millions  of  pages  of  written  data  that 
must  be  controlled,  cataloged,  updated,  stored,  and 
maintained.  In  addition,  a  change  in  one  category  of  TO  can 
easily  cause  numerous  changes  in  other  TO  publications  (3:1- 
1,  2-8) .  For  a  sophisticated,  cost  conscious  Air  Force,  a 
paper  based  TO  system  has  become  too  labor  intensive  and  too 
costly  to  support.  What  the  Air  Force  needs  is  a  Technical 
Order  system  that  utilizes  state-of-the-art  technology  to 
store,  update,  and  impart  technical  information  in  a  manner 
and  form  easily  used  and  understood  by  the  technicians 
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responsible  for  operating  and  maintaining  these  complicated 
weapon  systems  (4:2-1;  3-1).  During  the  last  thirty  years, 
the  Air  Force  has  adopted  several  computerized  data 
collection  systems  to  reduce  TO  processing  costs  and  improve 
information  updating  efficiency.  In  1961,  the  Air  Force 
upgraded  its  manual  TO  system  to  a  punch  card  batch  process 
system  called  the  Logistics  Management  of  Technical  Orders 
System  (GO-22)  and  converted  engineering  drawings  to  an 
aperture  card  (microfilm)  system  (4:3-2).  This  improvement 
was  considered  innovative  for  its  time  and  significantly 
reduced  TO  processing  time  and  storage  problems.  But  this 
system  soon  became  inundated  with  information  through  the 
acquisition  of  more  complex  weapon  systems.  This  system  was 
also  time  consuming  and  could  not  keep  pace  with  the  volume 
of  material  produced  by  these  new  weapon  systems  (5:6-10). 

The  next  major  conversion  took  place  in  March  1987  when 
the  Air  Force  installed  the  Automated  Technical  Order  System 
(ATOS) .  As  originally  conceived,  ATOS  would  be  implemented 
in  three  phases.  Phase  I  would  provide  each  Air  Force 
Logistics  Center  with  the  capability  to  make  page  changes  to 
TOs  on  a  real  time  basis.  Phases  II  would  establish  a  TO 
management  infrastructure  that  would  manage  all  Air  Force 
TOs  and  their  methods  of  distribution.  Phase  III  was 
designed  to  provide  real-time  access  and  distribution  of  TO 
information  at  base  level.  Phase  I  was  implemented,  but 
Phases  II  and  III  were  not,  due  to  budgetary  constraints  and 
technical  difficulties.  The  present  ATOS  system  can  only 


make  single  page  changes  to  TOs.  It  also  serves  as  a 
storage  and  retrieval  system  and  contains  a  system 
interface,  a  data  tagging  system  file,  and  a  data  dictionary 
(6:10-2;  7:3-1, 3-3;  8:1;  4:3-2). 

During  this  same  time  frame,  the  Air  Force  also  began 
digitally  storing  engineering  information  extracted  from 
aperture  cards  (microfilm)  under  the  Engineering  Data 
Computer  Assisted  Retrieval  System  (EDGARS) .  Although  both 
systems  increased  efficiency  while  decreasing  labor  and 
other  overhead  cost,  they  were  sub-optimal  and  could  not 
keep  pace  with  huge  amounts  of  information  generated  by  new 
weapon  system  acquisitions  (8:2). 

Currently,  the  Department  of  the  Air  Force,  in 
conjunction  with  its  sister  services  and  the  Department  of 
Defense  (DOD) ,  is  concentrating  on  a  system  philosophy  that 
would  standardize  the  conversion  and  acquisition  of  all 
technical  order  information  industry  wide.  Since  1987,  the 
Air  Force  has  initiated  several  TO  management  programs 
designed  to  create  a  modern  TO  management  infrastructure. 

The  Computer-aided  Acquisition  and  Logistic  Support 
(CALS)  initiative  developed  industry  standards  to  convert 
the  paper  intensive  TO  system  into  a  streamlined  digitally 
based  system.  This  system  affected  documentation, 
generation,  and  storage  of  technical  information  at  all 
management  levels  of  the  Air  Force  and  was  designed  to 
provide  operational  and  maintenance  personnel  with  technical 
information  on  a  real  time  basis  (9:3-9). 
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The  second  initiative,  termed  the  Air  Force  Technical 
Order  Management  System  (AFTOMS)  was  commissioned  to 
streamline  the  existing  Air  Force  TO  system  through  creation 
of  an  infrastructure  capable  of  managing,  processing, 
storing,  and  distributing  digitized  technical  information 
(4:4-2).  In  1990,  AFTOMS  became  a  joint  service  program 
under  the  Joint  Uniform  Service  Technical  Information  System 
(JUSTIS)  initiative.  JUSTIS  will  continue  with  the  CALS 
concept  and  further  develop  a  core  program  for  complete 
modernization  of  the  TO  system  (10:2-93,  2-100). 

Although  these  new  automated  systems  and  management 
initiatives  have  revolutionized  TO  documentation  and  storage 
techniques,  they  do  not  easily  interface  with  each  other 
which  makes  coordination  difficult.  Furthermore,  present- 
day  technicians  are  still  forced  to  use  antiquated,  time 
consuming,  and  hard-to-use  paper  instruction  manuals  to 
accomplish  maintenance  tasks  because  the  Air  Force  does  not 
have  the  capability  to  digitally  store  and  transmit 
technical  information  to  operational  units  and  maintenance 
facilities.  For  example,  two  System  Program  Offices,  the  F- 
22  and  B-2  SPOs,  under  the  authority  of  their  Program 
Manager  have  the  capability  to  store  and  process  the 
required  technical  information  from  respective  defense 
contractors  in  digital  form,  but  this  information  must  be 
converted  into  paper  TO  form  before  dissemination  throughout 
the  Air  Force. 
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since  1964,  the  Air  Force  has  also  conducted  several 
research  projects  in  an  effort  to  improve  technical  order 
content.  The  most  significant  of  these  projects  was  the 
Presentation  of  Information  for  Maintenance  and  Operations 
(PIMO)  project  conducted  by  Serendipity,  Incorporated  during 
1964-1969  (5:8-10;  11:13-15).  The  PIMO  project  researched 
and  developed  procedure lized  instruction  manuals  for  all 
maintenance  tasks  except  troubleshooting  tasks.  These 
instruction  manuals  provided  technicians  with  short,  easy- 
to-read  instructions  in  the  form  of  a  small  pocket  sized 
manual.  Between  1969  and  1972,  the  Armstrong  Laboratory 
(AL)  (formerly  the  Air  Force  Human  Resources  Laboratory),  at 
Wright-Patterson  Air  Force  Base,  Ohio,  in  conjunction  with 
the  Air  Force  Logistics  Command  (AFLC)  and  civilian 
contractors,  experimented,  tested,  and  refined  these  new 
pocket  sized  instruction  manuals  for  use  on  the  flight  line. 
Later,  troubleshooting  techniques  were  also  introduced  in 
paper  based  manual  form.  In  1972,  the  Air  Force  began 
procurement  of  these  small  instruction  manuals,  calling  them 
job  guides.  Job  guides  were  first  implemented  during  1976 
in  the  Military  Airlift  Command  (MAC)  for  the  newly  acquired 
C-141  Star lifter  weapon  system  and  were  later  developed  for 
all  remaining  aircraft  weapon  systems.  Now,  job  guides  are 
used  routinely  by  technicians  throughout  the  Air  Force 
(5:12-25;  12:1). 
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In  the  late  1970s  and  early  1980s,  computer  technology 
discoveries  (micro-chips)  reduced  the  costs  of  main  frame 
computers  and  led  to  the  production  of  desk-top  computers. 

A  short  time  after  their  development,  the  commercial  sector 
began  marketing  and  selling  desk-top  computers  to  commercial 
businesses  and  private  individuals.  Shortly  thereafter, 
these  technological  advancements  made  their  way  into  the 
defense  environment  and  the  Air  Force  began  routinely  using 
main  frame  and  desk- top  computers  to  conduct  daily  base- 
level  and  world-wide  business.  Recent  computer-based 
systems  have  aided  the  Air  Force  to  overcome  the 
restrictions  of  their  paper  system  reducing  the  complex 
departmental  problems  which  once  plagued  them.  The  Air 
Force  now  has  the  means  to  collect,  store  and  process  huge 
amounts  of  Information  in  a  timely  manner  which  has  yielded 
better  control  and  much  needed  standardization.  However, 
problems  soon  developed  became  departmental  and 
organizational  computer  systems  could  not  effectively 
interface  with  one  another.  A  few  examples  are:  the  Core 
Automated  Maintenance  System  (CAMS)  which  stores  technician 
training  information;  the  Standard  Base  Supply  System  (SBSS) 
that  processes  supply  requisitions;  and  the  Comprehensive 
Engine  Maintenance  System  (CEMS  IV)  that  tracks  and  stores 
information  on  aerospace  engines.  The  Air  Force  soon  found 
that  it  could  no  longer  quickly  update,  print,  and 
distribute  technical  information  in  TO  form  to  their 
operators  and  their  technicians.  Thus,  although  the 
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information  was  readily  available,  operators  and  technicians 
had  difficulty  gaining  access  to  it  (12:5). 

Armstrong  Laboratory  has  been  involved  with  testing  and 
developing  a  base-level  computer  system,  the  Integrated 
Maintenance  Information  System  (IMIS) ,  which  will  act  as  the 
interface  between  these  different  computer  systems.  The 
IMIS  system  improves  the  capabilities  of  base-level  aircraft 
maintenance  organizations  by  providing  technicians  a  single 
computerized  information  management  system  for  intermediate 
and  organizational  maintenance  (13:1).  In  fact,  a  recent 
survey  of  maintenance  technicians  conducted  by  the  AL  found 
that  a  computerized  display  of  technical  information  was 
preferred  over  the  paper  TO  system  currently  in  use  by  seven 
of  the  twelve  technicians  tested  (13:55-56).  The  IMIS  is 
providing  one  answer  to  the  access,  integration,  and 
information  display  problems  plaguing  the  Air  Force. 

It  [the  IMIS]  will  provide  the  technician 
direct  access  to  several  maintenance  information 
systems  and  data  bases  including  historical  data 
collection  and  analysis,  supply,  technical  orders, 
and  automated  training  systems.  (5:1) 

The  technician's  primary  interface  with  the  IMIS  is  a 

compact,  lightweight  battery-powered  portable  computer  that 

is  rugged  enough  for  flight  line  use  (5:2).  Although  much 

work  has  been  accomplished  since  1977,  when  this  idea  was 

first  conceived,  there  is  still  a  tremendous  amount  of 

experimentation  and  research  yet  to  be  accomplished. 
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The  next  phase  of  the  IMIS  innovation  process  is  to 
refine  the  information  transferral  process  to  its  user  by 
improving  the  design  of  its  hardware,  software  and 
human/ computer  interface.  The  AL  has  been  enhancing  the 
IMIS  computer  device,  its  software,  and  user  computer 
interfaces  for  almost  12  years.  Their  laboratory 
continually  develops  new  prototype  hardware  to  test  the 
latest  concepts  and  technologies.  Prototypes  are  evaluated 
by  users  under  field  demonstrations  and  the  results  are 
documented  in  hardware  and  software  specifications. 
Specifications  are  then  transitioned  to  implementing 
commands.  This  study  will  support  IMIS  hardware  and  user 
interface  definition  by  determining  the  best  cursor  control 
method (s)  to  be  used  to  access  information  from  a  compact- 
ruggedized  flight  line  computer.  The  information  gained 
from  this  experiment  will  drive  the  hardware  design  of  the 
IMIS  prototype  device  and  serve  as  the  standard 
specification  for  System  Program  Offices  (SPO)  as  they 
procure  portable  maintenance  aids  for  their  aircraft. 

The  management  problem  was,  "Of  the  different  indirect 
computer  access  methods  available  today,  which  method  is 
best  for  the  flight  line  environment." 

Research  OMegtiOh 

The  methods  available  to  access  computerized 
information  are  bunched  into  two  groups;  those  which  provide 
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direct  control  (a  light  pen  or  touch  sensitive  screen)  on 
the  screen  surface  and  those  which  provide  indirect  control 
(cursor  keys,  joystick  or  mouse)  away  from  the  screen 
(6:238).  The  flight  line  is  not  conducive  to  the  direct 
control  methods  currently  available  because  these  methods 
require  spotless  environments.  During  the  literature  review 
of  indirect  methods  summarized  in  Chapter  II,  the  mouse, 
trackball,  and  voice  entry  were  eliminated,  but  two  other 
access  methods  were  chosen  to  be  tested,  a  joystick  and 
four-cursor  keys.  A  joystick  normally  moves  the  cursor  in 
any  of  eight  directions  in  response  to  a  corresponding 
movement  by  the  computer  user.  Four-cursor  keys,  one  each 
pointing  up,  down  left  and  right,  move  the  cursor  one  step 
in  the  direction  of  the  key  when  it  is  depressed.  The 
research  question  which  emerged  was  "Which  of  two  methods, 
the  joystick  or  four-cursor  keys,  enables  the  user  to  access 
technical  information  from  a  ruggedized  computer  device  the 
fastest  and  provides  the  most  user  satisfaction?" 

Investigative  and  Measurement  Questions 

Based  upon  the  intent  of  our  research,  the  following 
areas  were  investigated: 

1.  Is  there  any  difference  between  access  times  for 
either  method? 

2.  Which  access  method  provides  the  most  satisfaction 
to  the  user/technician? 
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3 .  Is  there  a  correlation  between  the  fastest  method 
and  user  satisfaction? 

4.  Does  the  overall  access  time  vary  between  novice 
technicians  and  those  who  have  at  least  1  year  of  experience 
using  computers? 

Respondent  comments  were  solicited  using  a 
questionnaire  and  a  personal  interview  to  determine  each 
subject's  satisfaction  level  with  each  access  method  tested. 
Some  questions  dealt  with  each  respondent's  level  of 
experience  and  computer  proficiency  and  compared  them 
against  access  method  completion  times. 

Scope  dud  Limitations 

Every  attempt  was  made  to  make  this  experiment  as 
realistic  as  possible,  however  certain  limitations  were 
encountered.  The  joystick  device  and  software  programming 
restricted  the  cursor  movement  to  only  four  directions  and 
would  not  allow  diagonal  screen  movement  of  the  cursor. 

Also,  due  to  hard  drive  malfunctions,  the  PMA  had  to  be 
reconfigured  and  connected  to  a  UNISYS  desk  top  computer 
using  the  keyboard  terminal  jack  located  at  the  rear  of  the 
computer.  Consequently,  only  the  keyboard  funccion  of  tne 
PMA  was  used  in  the  test.  This  reconfiguration  did  not 
affect  the  validity  of  the  experiment  because  the  control 
devices  were  being  tested,  not  the  functionality  of  the  PMA. 

This  experiment  did  not  take  place  on  the  flight  line. 
Instead,  subject  were  observed  operating  the  PMA  keyboard 
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and  the  UNISYS  computer  within  the  confines  of  a  small 
office  located  in  the  maintenance  complex  of  the  4950th  Test 
Wing.  These  limitations  will  be  discussed  in  further  detail 
in  Chapter  III. 

Chapter  III,  Methodology,  describes  the  details  of  the 
experiment  which  was  designed  to  determine  the  best  indirect 
computer  access  method  to  use  in  a  maintenance/flight  line 
environment.  The  study  observed  technicians  performing  a 
maintenance  task  using  one  access  method  first  followed  by 
the  other  method.  During  the  experiment,  an  observer 
recorded  the  amount  of  time  to  complete  task  phases  and  the 
completion  time  of  the  overall  task.  Overall  task  times 
were  divided  into  phases  so  that,  if  necessary,  a  more  in- 
depth  analysis  could  be  made  by  the  researchers.  Chapter  IV 
describes  experimental  results  and  compares  them  against  the 
experimental  hypotheses.  Chapter  V  lists  our  conclusions 
and  recommendations  for  follow-on  experiments. 

Although  previous  research  has  examined  access  methods, 
their  studies  did  not  specifically  evaluate  performance  of 
indirect  access  devices  in  relationship  to  user  satisfaction 
and  their  practical  use  in  a  flight  line  environment.  This 
is  the  first  research  experiment  designed  to  test  more  than 
the  correlation  between  access  methods  and  cognitive 
learning  ability  and  skill. 
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II.  Literature  Review 


Background 

Technical  orders  (TOs)  have  undergone  considerable 
changes  since  they  were  introduced  as  single  page  "letters 
of  instruction"  in  the  Army  Signal  Corps  in  1918  (14:1). 
According  to  Air  Force  Regulation  8-2,  a  TO  constitutes  a 
military  order  and  is  issued  in  the  name  of  the  Chief  of 
Staff,  USAF,  by  order  of  the  Secretary  of  the  Air  Force. 
Virtually  every  system,  subsystem,  and  item  of  equipment  or 
support  equipment  must  be  operated  and  maintained  according 
to  the  procedures  contained  in  TOs  (2:2).  There  are  nearly 
20  million  pages  of  TOs  maintained  in  the  Air  Force 
inventory  (3:2-94).  This  does  not  include  technical  manuals 
maintained  by  civilian  contractors.  A  typical  TO  averages 
between  100  and  150  pages  and  a  simple  electronic  radio 
system  can  easily  require  over  15  separate  TOs  to  operate, 
maintain,  and  repair  its  systems.  On  larger  weapon  systems, 
the  B-1  for  example,  the  number  of  TOs  required  can  jump  to 
nearly  4000  (10:1). 

As  modern  weapon  systems  became  more  complex, 
technicians  began  criticizing  the  information  contained  in 
TOs.  Usually  technicians  found  TOs  hard  to  use,  and  the 
information  contained  therein  difficult  to  understand  (1:5; 
12:16-23).  It  soon  became  apparent  that  either  the  TO 
format  would  have  to  be  improved  or  another  information 
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medium  would  have  to  be  employed.  This  prompted  research  to 
study  the  possibilities. 

In  August  1962,  a  study  completed  by  Losee,  Allen,  and 
Stroud  of  the  AL  determined  that  the  Air  Force  TO  system 
hampered  (TOs  were  cumbersome  to  use  and  data  was  not  always 
accurate)  technician  perfoirmance  and  would  ultimately  lead 
to  degradation  of  weapon  system  integrity.  The  study  found 
that  the  information  contained  in  TOs  was  fragmented, 
inaccurate,  hard  to  read,  and  lacked  specific  guidance 
(5:7).  The  results  of  this  study  laid  the  foundation  for 
numerous  follow-on  research  projects,  one  of  which  was 
completed  in  1969  by  Serendipity  Incorporated. 

The  Serendipity  project,  called  the  Presentation  of 
Information  for  Maintenance  and  Operations  (PIMO)  project, 
led  to  the  development  of  proceduralized  instruction  manuals 
for  all  maintenance  tasks  except  troubleshooting  tasks. 

These  proceduralized  manuals  provided  technicians  with 
condensed,  easy-to-read  instructions  in  the  form  of  small 
pocket-sized  manuals.  Later  refinements  and  standards 
developed  by  AL  (Joyce,  Laid,  Mulligan,  and  Mallory  in  1973) 
coupled  with  several  field  tests  finally  led  to  the 
implementation  of  these  manuals  between  1972  and  1976  as  job 
guides  (JG)  (5:8-13).  In  1975,  Dr.  Donald  Thomas  of  the 
Armstrong  Laboratory,  conducted  another  opinion  survey  of 
Air  Force  maintenance  personnel.  These  modern  airman  stated 
that,  "Technical  Orders  are  difficult  to  use,  poorly 
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organized,  and  written  in  temninology  that  is  difficult  to 
understand”  (1:7) . 

Since  the  early  1960s,  the  Air  Force  has  implemented 
several  computerized  data  collection  systems  in  an  effort  to 
improve  TOs.  In  1961,  it  activated  a  batch  process  system 
called  the  Logistics  Management  of  Technical  Orders  System 
which  converted  engineering  drawings  to  microfilm.  Then  in 
1987,  the  Air  Force  installed  the  Automated  Technical  Order 
System  (ATOS)  which  allowed  them  to  make  page  updates  to  TOs 
on  a  real  time  basis  while,  at  the  same  time,  serving  as  a 
storage,  retrieval,  and  interface  system  (3:2-97). 

Later,  the  Air  Force  extended  ATOS  to  digitize  its  TOs 
which  decreased  costs  and  improved  update  and  delivery 
procedures.  This  change  has  progressed  satisfactorily,  but 
ATOS  will  fall  short  of  its  desired  goal,  "to  eventually 
provide  totally  accurate  and  timely  TOs  for  use  by  Air  Force 
technicians"  (14:10-26).  The  causes  that  led  to  the  failure 
of  ATOS  were  the  extended  length  of  time  it  took  to  update 
the  information  stored  in  the  database  (convert  information 
from  digital  form  to  a  paper  instructional  form)  and  the 
difficulty  in  deciding  which  procedure  (paper  or  automated 
TOs)  should  be  used  to  present  and  access  the  information. 

As  weapon  systems  become  more  complex,  the  information 
necessary  to  operate,  maintain,  and  repair  them  begins  to 
expand  geometrically.  What  the  modern  technician  needs  is  a 
system  which  integrates  information  from  many  different 
computers  into  one  retrieval  system  that  is  accurate,  simple 
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to  use,  easily  accessible,  and  easy  to  update.  Paper  TOs  no 
longer  meet  this  demand  and  alternate  Information 
dissemination  methods  are  required  If  technicians  are  to 
perform  their  jobs  In  a  timely  and  responsible  manner. 

Development  of  the  computer  micro-chip  and  later 
refinements  In  micro-chip  technology  In  the  1970s  opened  new 
avenues  for  collection  and  circulation  of  Information. 
Computers  were  miniaturized  and  computer  storage  capacity 
was  greatly  expanded  which  led  to  the  creation  of  mini-  and 
micro-computers.  Micro-computer  technology  opened  a 
revolutionary  means  to  gather  and  distribute  technical 
Information  and.  In  1977,  under  contract  with  Behavioral 
Technology  Consultants  of  Silver  Spring,  Maryland,  the 
Armstrong  Laboratory  began  testing  micro-computer  technology 
as  a  means  to  automate  the  Air  Force  TO  system  (15:6).  In 
1979,  Frazier  of  Behavioral  Technology  Consultants  completed 
a  study  which  produced  the  following  conclusions: 

(1)  Multiple  Information  levels  should  be  provided  to 
accommodate  the  varying  experience  levels  of  maintenance 
technicians. 

(2)  All  Information  should  be  filed  In  one  place  to 
allow  easy  access  and  duplication. 

(3)  Menu  options  should  be  provided  to  allow  a 
technician  to  access  ancillary  Information  Involving  test 
equipment  operating  procedures,  etc. 

(4)  Information  should  be  provided  In  a  step-by-step 
format  and  used  In  conjunction  with  graphic  Illustrations. 
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(5)  Graphic  representations  should  be  as  simple  as 
possible  for  ease-of-use  and  reduction  of  computer  storage 
and  retrieval  times. 

(6)  Special  keys  should  be  used  to  reduce  the 
requirement  for  extensive  typing  skills  (15:6-7). 

Following  this  study,  AL  began  developing  and  testing 
micro-computer  based,  automated  TO  display  devices  under 
actual  flight  line  conditions.  Developmental  studies  were 
conducted  on  a  contractual  basis  with  private  contractors, 
internally  within  the  departments  of  the  AL,  and  in 
partnership  with  agencies  within  other  DOD  departments. 

In  1984,  the  AL  determined  that  sufficient  in-house 
experiments  had  been  conducted  and  it  was  time  to  test  a 
computer-based  maintenance  aiding  system  (CMAS)  under  field 
conditions.  A  test  was  scheduled  at  Offutt  Air  Force  Base, 
Nebraska,  in  December  1984,  to  examine  the  practicality  and 
user  satisfaction  of  automated  TOs.  Strategic  Air  Command 
(SAC)  technicians  put  the  system  through  its  paces.  The 
results  of  this  test  were  somewhat  disheartening,  as  the 
performance  of  CMAS  I  failed  to  operate  to  technician's 
satisfaction.  The  system  was  too  large  and  the  computer 
response  time  too  slow  for  routine  use  (15:7-8). 

Returning  to  the  drawing  board,  the  Armstrong 
Laboratory  st;r^  ed  the  development  of  CMAS  II  which  made 
significant  imp.  cements  to  CMAS  I  based  upon  results 
obtained  from  the  1984  study.  CMAS  II  was  smaller  in 
design,  contained  a  greatly  expanded  memory,  and  provided 
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technicians  with  a  menu  system,  function  keys,  and  cursor 
keys  that  were  incorporated  on  a  single  keyboard.  Another 
operational  test  was  scheduled  at  Grissom  Air  Force  Base, 
Indiana,  and  the  results  of  this  operational  test  were 
tremendously  successful.  Technicians  found  maintenance 
information  to  be  easier  to  access  and  quicker  to  extract 
using  the  CMAS  II  as  compared  to  researching  the  same 
information  in  a  paper  TO.  Criticism  of  the  CMAS  II  laid  in 
its  inability  to  display  an  entire  schematic  diagram. 
Technicians  became  frustrated  when  trying  to  determine  the 
source  of  an  equipment  malfunction  because  the  computer 
screen  only  displayed  small  segments  of  the  total  schematic 
diagram.  When  tracing  signal  paths,  technicians  were  easily 
lost  during  the  tracing  process  because  they  had  to  access 
several  different  computer  screens  to  determine  the  total 
signal  path.  Once  lost,  the  technician  had  to  begin  the 
signal  tracing  process  over  again.  Except  for  this 
inconvenience,  technicians  accepted  the  CMAS  II  as  a  viable 
alternative  to  paper  TOs  (15:8-9). 

In  1987,  the  Navy  Personnel  Research  and  Development 
Center  at  San  Diego,  California,  performed  tests  using  CMAS 
II  to  troubleshoot  and  fault  identify  a  radio  receiver- 
transmitter  (16:2-4).  The  study  concluded  that. 

Computers  can  be  used  as  an  effective  means  to 
present  technical  information  in  an  electronic  format 
[and]  more  Importantly,  technicians  appear  willing  to 
change  to  a  different  delivery  method  for  obtaining 
maintenance  information.  (16:12) 
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since  1987,  the  CMAS  has  been  developed,  refined,  and 
expanded,  and  is  now  known  as  the  Integrated  Maintenance 
Information  System  (IMIS)  program.  This  program  has  led  to 
the  development  and  testing  of  a  technician's  portable 
maintenance  aid  (PMA)  that  will  not  only  improve  a 
technician's  access  to  weapon  system  information,  but  will 
interface  with  other  computer-based  systems  currently  used 
by  the  Air  Force  in  its  daily  business  routines.  In  1991, 
the  AL  sponsored  an  experiment  comparing  the  usefulness  of  a 
Head  Mounted  Monocular  Device  and  the  PMA  to  troubleshoot  a 
malfunction  in  a  radio  transceiver.  After  evaluating  16 
technicians  using  these  devices,  it  was  determined  that  no 
significant  difference  between  the  two  devices  existed,  as 
both  computer  display  devices  worked  equally  well  for 
technicians  to  accomplish  maintenance  routines  (17:57).  The 
Armstrong  Laboratory  has  concentrated  their  research  efforts 
on  developing  a  IMIS  PMA  that  would  provide  technicians  with 
immediate  and  timely  access  to  information  necessary  for  job 
accomplishment.  This  research  effort  led  to  the  development 
of  a  PMA  sturdy  enough  for  the  flight  line  environment, 
versatile  enough  to  display  information  in  an  easy-to-use 
format,  and  flexible  enough  to  provide  several  information 
display  levels  to  accommodate  differing  levels  of  technical 
experience  (13:35-36).  Now  that  a  computer  generated  TO  has 
proven  to  be  technologically  practical,  the  AL  is  also 
directing  its  research  efforts  toward  evaluation  of  computer 
access  method (s)  and  the  design  of  human  computer 
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interfaces.  At  this  point  in  time,  direct  access  methods 
(touch  screen,  light  pen)  are  not  rugged  enough  to  survive 
the  chemical  severity  of  the  flight  line  (dirt,  oil, 
cleaning  solvents,  etc.).  Although  indirect  access  methods 
are  more  rugged  than  direct  access  methods,  some  indirect 
access  methods  are  impractical  for  use  on  a  flight  line. 
Therefore,  the  important  question  to  be  answered  through 
this  computer  access  research  is,  "Which  of  the  indirect 
access  methods  can  endure  the  flight  line  environment  the 
best,  while  providing  technicians  with  easy  access  to 
information?"  Another  important  and  possibly  overriding 
factor  to  be  studied  is,  "Which  indirect  access  method  is 
preferred  by  the  technician?"  The  rest  of  this  literature 
review  explores  the  vast  array  of  computer  access  methods 
currently  available  and  attempts  to  determine  their 
individual  advantages  and  disadvantages.  The  review  then 
narrows  its  focus  to  indirect  access  methods  and  their 
capacity  to  manipulate  technical  information  in  relation  to 
their  use  on  the  flight  line. 

Integrated  Maintenance  Information  System 

Current  data  processing  innovations  have  enabled  the 
automation  of  many  maintenance  processes. 

These  processes  include  interacting  with 
maintenance  databases,  [retrieving]  and  using  technical 
[ information] ,  and  downloading  and  evaluating  aircraft 
built-in-test  to  diagnose  malfunctions  in  electronic 
and  computer  hardware.  (13:1) 
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A  number  of  computerized  systems  (CAMS,  SBSS,  CEMS  IV) 
under  development  or  already  implemented  by  the  Air  Force 
will  aid  technicians  to  perform  their  tasks  more  efficiently 
in  the  future.  These  projects  were  designed,  developed,  and 
managed  separately,  leading  to  different  hardware  and 
human/ computer  interfaces  for  each  system.  Thus,  automated 
tools  designed  to  make  a  technician's  job  easier  actually 
made  it  more  complex  because  technicians  had  to  use  several 
computer  systems.  What  technicians  need  is  a  single 
integrated  computer  system  that  improves  the  technical  data 
retrieval  process  and  easily  integrates  with  other  computer 
systems  (13:1). 

The  AL  has  developed  the  Integrated  Maintenance 
Information  System  (IMIS)  to  meet  this  need. 

The  IMIS  uses  a  portable  computer  to  extract 
information  from  the  aircraft  and  provides  a  single 
interface  for  information  from  CAMS,  JUSTIS,  SBSS, 
REMIS,  wees,  CEMS  IV  and  any  other  information  system 
required  to  support  maintenance  activities  on  the 
flight  line  and  in  the  maintenance  shop.  (17:20) 

Figure  2  displays  these  information  systems  and  the  current 

total  interface  scheme. 


Source :  Masque 1 ier 


Figure  2.  Integrated  Maintenance  Information  System 
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The  IMIS  contains  five  major  elements: 

-  The  technicians  portable  maintenance  aid. 

-  A  maintenance  Information  work  station. 

-  An  aircraft  Interface  panel  link  with  computers. 

-  Integration  software  to  present  data  consistently. 

-  Applications  software  to  troubleshoot  and  identify 
the  cause  of  malfunctions  and  perform  maintenance  (13:1-2). 
The  current  portable  maintenance  aid  (PMA)  is 

A  portable,  ruggedized  computer  which  operates 
using  a  battery  pack  or  the  aircraft's  system  power  and 
acts  as  the  single  access  unit  for  all  the  data  needed 
to  accomplish  maintenance  tasks.  (13:1,4) 

The  technique (s)  used  to  organize  and  display  this 

information  can  determine  the  success  of  the  system. 

Accordingly,  AL  personnel  emphasized  the  importance  of  using 

the  best  data  display  and  access  techniques  possible  (13:3). 

To  test  the  utility  of  the  IMIS,  the  AL  performed  a 

diagnostic  demonstration  on  a  F-16  fire  control  radar  using 

twelve  maintenance  personnel  from  Homestead  AFB,  Florida  for 

a  two-week  period  (13:20).  After  a  brief  training  period, 

each  pair  of  subjects  performed  built-in-tests  (BIT)  to 

identify  a  problem,  correct  the  problem,  and  then  verify  the 

repair  using  BIT  again  (13:22).  After  the  demonstration, 

the  outcome  confirmed  that  the  IMIS  performed  as  expected. 

In  fact,  questionnaires  employed  by  the  AL  revealed  that 

most  subjects  rated  the  key  operation,  response  time,  screen 

size,  character  legibility,  white  space,  and  overall  time 

and  effort  required  to  operate  the  PMA  as  very  satisfactory 
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or  better  (13:31).  Also,  seven  of  the  twelve  subjects 
stated  that  they  would  prefer  the  prototype  (test)  computer 
to  paper  TOs  (13:55-56).  During  the  course  of  this  thesis 
research,  the  AL  conducted  more  experiments  with  improved 
versions  of  the  IMIS  device  which  supported  the  F-18 
demonstration  computer  preference  choice.  Result  of  those 
demonstrations  were  not  published  to  be  included  in  this 
literature  search. 

Since  the  Armstrong  Laboratory  has  established  the 
feasibility  of  the  IMIS  computer  system,  this  study  attempts 
to  find  the  best  pointing  method (s)  to  access  information 
from  a  flight  line  computer.  The  methods  available  to 
access  computerized  information  can  be  clustered  into  two 
groups;  those  which  provide  direct  control  on  the  screen  and 
those  which  allow  indirect  control  away  from  the  screen 
(6:238).  Lightpens  and  touch  screens  can  directly  access 
information  by  touching  the  appropriate  symbol  or  position 
on  the  computer  screen  with  the  lightpen  or  finger 
respectively.  These  methods  cause  the  user's  hand  to 
obscure  part  of  the  screen  and  need  the  screen  to  remain  oil 
and  dirt  free  (6:240).  Direct  access  methods  would  not  work 
well  for  maintenance  tasks  because  technicians  need  constant 
exposure  to  pertinent  information  and  the  flight  line  area 
is  not  clean.  According  to  Schneiderman,  several  indirect 
pointing  methods  including  a  joystick,  a  mouse,  and  a 
trackball,  could  retrieve  data  from  the  IMIS.  Four-cursor 
keys  and  voice  entry  could  also  be  used  (6:236-244). 
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Several  authors  have  performed  experiments  with 
computers  utilizing  these  devices  to  access  different  types 
of  information  and  published  their  conclusions.  Summaries 
of  the  research  and  their  findings  follow: 

Table  1  summarizes  these  cursor  control  devices,  their 
disadvantages  and  their  recommended  uses  (18:154). 

TABLE  1 

CONTROL  DEVICES 


DEVICE 

USES 

(CURSOR) 

DISADVAMTAOES 

RECOMMENDED 

FOR 

NOT  RECOMMEND 
FOR 

Joystick 

Track/ 

select/ 

move 

Mouse  may  be 
faster  to 
select  text 

Tasks  with 
intensive 
positioning 

Frequent 
chgs  to/ from 
keyboard 

Mouse 

Point/ 

select/ 

draw/drag 

Needs  desk 
space  S  has 

trailing  cord 

Task  needing 
little  key¬ 
board  use 

Frequent 
mouse  6  key¬ 
board  changes 

Trackball 

Track/ 

select/ 

move 

Mouse  may  be 
faster  to 
select  text 

Integrated 
graphics  & 
keyboard 

Frequent 
chgs  to/ from 
keyboard 

4  Cursor 
Keys 

Discrete 

cursor 

movement 

Slow  to 
move  cursor 
long  distance 

Tasks  needing 
short  cursor 
movements 

Extensive/ 
fine  cursor 
movements 

Voice 

Entry 

Enter  #8/ 
Start 
predefine 
actions 

Requires 
step-by-step 
confirmation 
of  entries 

When  hands 
or  eyes  are 
not  free 

Noisy  or 

stressful 

environments 

Source :  Brown 


Studies  conducted  by  the  University  of  York  provide  a 
detailed  description  of  these  access  devices. 

Joystick.  The  joystick  is  extremely  popular  to  pick 
and  position  a  cursor  on  a  cathode  ray  tube  (CRT)  screen 
(19:109).  Movement  of  the  joystick  causes  a  corresponding 
movement  of  the  cursor  in  the  same  direction.  Holding  the 
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joystick  in  one  position  continuously  causes  the  cursor  to 
move  in  that  direction  until  the  joystick  is  released.  The 
joystick  can  be  housed  in  the  computer  frame  or  attached  to 
the  computer  by  a  control  cable.  VThen  covered  with  a 
flexible  rubber  cover,  the  joystick  is  protected  from 
penetration  of  dirt,  moisture,  and  oil  which  would  cause 
failure. 

Mouse .  This  interface  device  is  pushed  along  a  clean, 
flat,  and  smooth  surface  near  the  computer.  Movement  of  the 
mouse  causes  a  corresponding  cursor  movement  in  the  same 
direction  and  distance.  This  device  is  very  popular  for 
picking  items  on  menus,  positioning  the  cursor  on  objects 
displayed  on  a  CRT,  and  for  drawing  lines  on  a  CRT  (20:109). 
The  mouse  is  not  suited  for  use  on  the  flight  line  because 
it  requires  a  clean,  flat,  oil-free  surface  to  slide  on. 

Trackball.  Movement  of  the  cursor  is  accomplished  by 
rotating  a  trackball  within  its  socket  to  move  the  cursor  on 
the  CRT.  The  cursor  can  be  moved  in  any  direction  using 
this  device  making  it  popular  for  picking  and  positioning 
(20:109).  Functionally,  the  trackball  is  similar  to  the 
mouse,  but  highly  prone  to  malfunction  when  exposed  to  dirt, 
moisture,  and  oil  based  contaminates.  The  trackball  would 
not  be  well  suited  for  use  on  the  flight  line  for  this 
reason. 

Four-Cursor  Kevs.  Cursor  keys  cause  cursor  movement  in 
a  left-right  or  up-down  direction  only,  and  are  extensively 
used  in  word  processing  and  templated  documents.  Novice 
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users  rely  upon  the  cursor  keys  whereas  experienced  users 
usually  employ  other  methods.  Most  cursor  keys  incorporate 
a  single  movement  function  and  an  auto-repeat  function.  In 
the  single  movement  function,  depressing  the  cursor  key 
moves  the  cursor  a  single  character  or  line,  versus  the 
auto-repeat  function,  where  holding  the  cursor  key  down 
causes  the  cursor  to  move  across  multiple  characters  or 
lines  (6:236-237). 

Voice  Entry .  Commonly  referred  to  as  "the  interface  of 
the  future” ,  voice  entry  employs  a  speech  recognition  system 
that  compares  spoken  words  to  voice  patterns,  or  single 
words  and  phrases  stored  in  memory.  These  reference 
patterns  are  entered  into  the  machine  a  word  or  phrase  at  a 
time  while  the  machine  is  in  the  training  mode.  Once  the 
words/phrases  have  been  entered  into  a  computer's  long-term 
memory,  the  computer  listens  to  spoken  words/phrases,  and 
then  compares  these  spoken  words/phrases  with  those  on  file. 
When  a  match  is  recognized,  a  predetermined  command  is 
executed.  Although  highly  effective  under  laboratory 
conditions,  voice  entry  technology  has  not  developed  to  the 
point  of  casual  use.  The  automatic  speech  recognizer  is 
easily  confused  by  background  noise,  individual  voice 
patterns/ accents,  and  minute  changes  in  people's  voice  over 
time  resulting  in  non-performance  or  incorrect  performance 
of  commands.  Also,  a  user's  normal  speech  habits  must  be 
altered.  The  user  must  speak  in  a  slow  and  deliberte 
manner,  pausing  after  each  word  or  phrase,  to  allow  the 
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speech  recognizer  time  to  listen,  compare,  and  then  respond. 
Although  an  usual  speech  pattern  is  not  impossible  to 
overcome,  it  becomes  extremely  frustrating  to  the  user  and 
results  in  counter  productivity  (20:37-41). 

Table  2,  summarizes  three  input  techniques  and  lists 
the  preferred  access  method (s)  to  perform  the  operation. 

TABLE  2 

PREFERRED  INPUT  DEVICES  FOR  DIFFERENT  INPUT  TECHNIQUES 

Picking;  mouse,  joystick/trackball,  light  pen/touch 

screen/soft  keys,  four-cursor  keys. 

Positioning:  mouse,  joystick/trackball,  light  pen/touch 
screen,  four-cursor  keys. 

Drawing:  mouse,  light  pen. 

(Highest  preferences  are  to  the  left  and  lowest  to  the 
right,  with  equal  preferences  separated  '/') 

Source:  University  of  York 


Access  Method  Selection 

Even  though  most  of  these  experiments  suggest  the  mouse 
performs  best,  we  eliminated  it  from  our  test  for  a  flight 
line  environment  because  it  requires  a  flat  surface  close  by 
to  allow  cursor  movement,  has  a  vulnerable  connecting  cord 
which  could  easily  get  tangled  or  ruined,  and  demands  a 
clean  environment.  The  trackball  also  needs  a  clean 
laboratory  type  environment  to  function  reliably,  so  it 
should  not  be  considered  for  flight  line  use  either.  In 
addition,  the  voice  entry  device  should  also  be  deleted 
because  of  the  excessive  noise  prevalent  during  maintenance 
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and  the  inadequate  technology,  causing  it  to  be 
undependable.  Therefore,  the  two  access  methods  tested  were 
the  joystick  and  four-cursor  keys. 

These  results  correlate  favorably  with  studies 
performed  by  the  Armstrong  Laboratory,  which  concluded  that 
a  joystick  or  four-cursor  keys  would  function  best  under 
flight  line  conditions. 
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Ill .  Methodology 


Introduction 

As  indicated  in  Chapter  1,  studies  are  still  underway 
to  determine  if  computer  generated  technical  orders  (TO)  can 
replace  paper  TOs.  Several  studies  have  indicated  that 
computer  generated  TOs  are  superior  to  paper  TOs  for  ease  of 
information  access  and  user  satisfaction.  The  purpose  of 
this  study  is  to  investigate  which  of  two  computer 
access/user  interaction  devices  provides  the  fastest  access 
to  information  contained  on  a  computerized  maintenance 
database  system,  and  which  device  provides  its  users  with 
greater  satisfaction.  Two  comparable  "simulated"  tasks  were 
selected  and  programmed  into  a  portable  computer  attached  to 
a  keyboard  provided  by  the  AL.  A  pilot  study  was  conducted 
on  both  tasks  to  ensure  they  met  all  the  experimental 
parameters  and  were  similar  enough  both  in  their  level  of 
difficulty  and  the  total  time  required  to  complete  all  the 
separate  taskings  to  preclude  any  bias.  The  experiment  took 
place  in  a  controlled  environment  within  the  maintenance 
complex  of  the  4950th  Test  Wing  at  Wright-Patterson  AFB  OH. 

Experimental  Design 

An  independent,  random  sample  of  32  technicians  (16 
experienced  and  16  inexperienced)  from  various  Air  Force 
specialties  employed  within  the  4950th  Test  Wing  was 
selected  to  participate  because  of  their  availability.  It 
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was  determined  that  maintenance  technicians  from  the  4950th 
Test  Wing  would  provide  a  good  cross-sectional  sample  of  the 
population  of  all  Air  Force  technicians  since  this 
experiment  investigates  which  access  method  results  in  the 
completion  of  two  comparable  tasks  sooner.  Each  technician 
performed  two  "simulated”  tasks  which  required  access  to 
maintenance  data  stored  on  the  portable  computer  using 
either  the  joystick  device  or  four-cursor  keys.  Data  was 
collected  and  analyzed  to  test  the  investigative  questions 
outlined  in  Chapter  1.  Following  the  experiment,  each 
respondent  completed  a  questionnaire  and  was  interviewed  to 
determine  their  level  of  satisfaction  with  each  device. 
Additional  information  and  suggestions  not  readily  apparent 
during  the  experiment  were  also  solicited  by  the 
researchers.  Appendix  A  contains  the  detailed  experimental 
plan  used  to  conduct  the  experiment  and  Appendixes  B  and  C 
comprise  detailed  listings  of  the  two  simulated  tasks.  See 
Appendix  D  for  the  background  survey  and  Appendix  E  for  the 
observation  form  used  to  collect  the  access  times  and  other 
information.  Appendixes  F  and  G  display  the  questionnaire 
and  interview  questions. 

Test  Hardware 

The  Integrated  Maintenance  Information  System  (IMIS) 
portable  maintenance  aid  (PMA) ,  was  originally  intended  to 
be  used  for  this  experiment.  The  IMIS  PMA  is  an  AL 
prototype  ruggedized  computer  designed  for  flight  line  use. 
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It  weighs  about  8.75  pounds  and  measures  11"  X  8.5"  X  2.5". 
It  houses  four-cursor  keys,  a  joystick,  ten  numeric  keys, 
six  function  keys  plus  several  other  dedicated  keys,  and  an 
8"x  6"  electroluminescent  screen.  However,  due  to  a 
scheduling  conflict  with  the  Armstrong  Laboratory,  a  UNISYS 
personal  computer  was  used  to  run  the  tasks  utilizing  only 
the  keys  on  an  attached  IMIS  maintenance  aid  case  (See 
Figure  3) .  Also,  the  layout  of  the  PMA  is  shown  in  Figure  4 
on  page  34. 


Figure  3.  Combination  UNISYS  Computer/IMIS  PMA  Device 
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Table  3  lists  some  pertinent  specifications  for  the  UNISYS 
portable  computer/IMIS  keyboard  combination. 

TABLE  3 

HARDWARE  SPECIFICATIONS  FOR  THE  UNISYS/ IMIS  COMPUTER 


Computer  Feature 
Random  Access  Memory 
Hard  Disk  Size 
CPU  Processor 
Math  Coprocessor 
Power  Source 


Hardware 
16  MBytes 
163  MBytes 
80386SX-20 
80387 
120  VAC 


Experimental  Tas.H.s 

Two  tasks  requiring  access  to  information  in  several 
forms  were  developed  for  this  experiment.  A  "simulated** 
Built-In-Test  (BIT)  routine  for  the  F-18  Nose  Wheel  Steering 
(NWS)  module  served  as  task  one  while  task  two  utilized  an 
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"artificially"  developed  Main  F-18  BIT  routine.  These 
routines  required  the  technicians  to  scroll  about  a  computer 
screen  to  access  and  use  different  types  of  information  to 
order  parts,  chose  items  from  a  menu,  perform  calculations 
using  a  screen  displayed  calculator  and  chose  replacement 
parts  from  a  parts  list.  Observations  of  technician  and 
computer  interaction  provided  an  excellent  means  to 
determine  the  fastest  method  for  accessing  information 
stored  in  the  IMIS.  Each  task  used  two  continuity  checks 
and  the  replacement  of  either  a  relay  or  a  connector  to 
present  technical  information.  These  "simulated"  tasks 
underwent  a  pilot  test  using  AFIT  students  and  were  refined 
prior  to  conducting  this  experiment. 

Software 

The  UNISYS  PC  uses  Microsoft  MSDOS,  version  5.0  as  its 
operating  system.  The  AL  programmed  the  tasks  using  Version 
1.1  of  Smalltalk  V  and  a  program  of  their  own  creation,  the 
F-18  Presentation  System.  Prior  to  this  experiment,  all 
necessary  information,  including  some  schematic  diagrams, 
was  loaded  into  the  UNISYS  computer.  Smalltalk  V  is  an  off- 
the-shelf  programming  product  and  was  chosen  because  of  its 
response  time,  and  its  ability  to  easily  maneuver  about  a 
computer  screen  (21) .  Smalltalk  V  has  three  features 
necessary  to  test  the  joystick  and  four-cursor  keys,  (1)  the 
ability  to  design  several  different  charts  and  diagrams,  (2) 
two  on-screen  cursor  select  menus,  and  (3)  the  ability  to 
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scan  through  multiple  screens  of  information  relatively 
quickly.  The  next  section  provides  details  on  the 
procedures  required  to  operate  the  IMIS  system  and  was 
extracted  from  the  “Common  User  Interface  Specification"  for 
the  F/A-18  Presentation  System,  Version  1.9,  published  by 
Logicon  Eagle  Technologies. 

IMIS  Primary  Maintenance  Aid  Cursor  Movement  Procedures 

Each  user  moved  the  cursor  in  one  of  two  manners:  by 
pushing  the  cursor-keys  or  by  manipulating  the  thiimb  knob 
(joystick)  device.  When  moving  the  cursor  directly,  a  user 
employed  a  leap  mode  which  worked  like  a  tab  key  to  induce 
the  cursor  to  "leap",  or  jvu&p,  to  the  next  selectable  item 
(22:10).  Thus,  the  cursor-keys  and  thumb  operated  knob 
provided  alternate  methods  to  perform  the  same  action — move 
a  cursor  in  one  of  four  directions  (up,  down,  left,  right) . 
The  two  control  devices  could  be  used  interchangeably  to 
produce  the  same  cursor  behavior.  The  AL  added  these  two 
controls  to  the  PMA  specifically  for  this  experiment  to 
evaluate  which  device  provided  the  most  effective  cursor 
control  technique  to  access  technical  information  (22:11). 
According  to  the  F-18  Presentation  Specification, 

The  cursor-keys  or  thumb  knob  (joystick)  may  also 
be  placed  into  a  Scroll  Mode,  which  temporarily 
disabled  cursor  movement  and  directed  all  cursor- 
key/th\amb  knob  actions  to  the  scrolling  of  a  text, 
graphic  or  table  pane.  (22:10) 

Cursor  Movement  Devices .  There  were  four  separate 
arrow  keys  which  moved  the  cursor  up,  down,  left  or  right 
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when  depressed.  The  thumb  knob  was  a  four  position  rocker¬ 
like  force  sensitive  stick  which  didn't  actually  move,  but 
responded  to  the  force  applied  by  the  user.  Each  position 
had  the  same  effect  as  the  matching  arrow  key  (22:12). 

Selectable  Items.  Each  selectable  item  on  the  screen 
had  a  "hot  spot",  or  selectable  region  which  highlighted 
(turned  reverse  video)  whenever  the  cursor  moved  into  that 
region  (22:10).  When  a  dialog  box  was  active  on  the  screen, 
the  available  selections  would  only  be  the  entry  fields  and 
the  push  buttons  displayed  within  the  box  (22:10). 

Cursor  Movement  between  Non-Adi acent  Regions .  The 
cursor  did  not  appear  on  screen  between  any  regions.  It 
always  "leaped"  using  a  tabbing  type  method  to  move  from  one 
"selectable"  region  to  another.  A  corner  stitch  of  two 
tabbing  schemes  was  used  for  each  display:  a  horizontal  and 
a  vertical  tab  (22:14).  The  horizontal  tabbing  scheme 
assigned  an  order  to  all  displayed  selectable  regions  using 
the  upper  left  corner  of  the  screen  as  an  anchor  point.  The 
regions  would  be  assigned  using  a  horizontal  pixel 
procedure,  moving  down  line  by  line  until  the  bottom  right 
corner  of  the  display  was  reached.  Also,  the  other 
(vertical)  tab  scheme  assigned  the  same  regions  using  a 
corresponding  vertical  pixel  method  until  the  identical 
corner  was  reached.  The  final  result  was  a  movement  system 
which  didn't  move  only  horizontally  or  vertically,  but  a 
combination  of  both  directions.  Obviously,  there  were  some 
deficiencies  with  the  final  "corner  stitched"  technique,  but 
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it  ensured  that  a  user  could  reach  every  available 
selectable  region  displayed  on  the  screen,  which  was  a 
problem  perceived  with  many  of  the  other  available  leaping 
modes  (22:14-16).  While  the  corner  stitch  system  did  ensure 
complete  access  to  the  screen,  it  remains  to  be  seen  whether 
this  pattern  will  effect  the  experimental  results  by 
adversely  influencing  the  joystick  (thumb  knob)  response 
times. 

Tasks 

The  F-18  Built-In-Test  preventive  maintenance  routines 
used  to  test  the  access  methods  were  selected  because  they 
required  repetitive  access  to  previously  stored  data  and 
information  to  perform.  Both  tasks  were  matched  in  regards 
to  complexity,  the  necessity  to  move  across  the  computer 
screen  several  times,  and  the  requirement  to  perform  cursor 
movements  within  selected  CRT  displays  (See  Appendices  B  and 
C) .  Both  tasks  involved  several  types  of  cursor  movements 
including  searching  for  and  selecting  a  string  of  text, 
using  programmable  function  keys,  moving  about  several  block 
diagrams  and  multiplying  two  numbers  using  a  screen 
displayed  cursor  based  calculator.  Each  subject  performed 
both  tasks  based  on  a  random  assignment  to  one  of  four 
groups.  To  help  eliminate  any  learning  curve  bias,  task  one 
and  the  joystick  were  used  first  by  only  half  of  the 
technicians.  To  be  precise,  group  one  performed  task  one 
first  using  the  joystick  and  task  two  with  the  four-cursor 
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keys  while  group  two  accomplished  task  two  first  with  the 
joystick  followed  by  task  one  with  the  four-cursor  keys. 

The  other  groups  employed  the  same  order  of  tasks,  but  used 
the  four-cursor  keys  first  (see  Table  4) . 

TABLE  4 

ACCESS  METHOD  AND  GROUP  ORDER  USED  FOR  SIMULATED  TASKS 

GROUP  NUM  FIRST  TASK  FIRST  METHOD  EXPERIENCE 

GROUP  1  Task  1  Joystick  4-Low/4-High 

GROUP  2  Task  2  Joystick  4-Low/4-High 

GROUP  3  Task  1  Cursor-Key  4 -Low/ 4 -High 

GROUP  4  Task  2  Cursor-Key  4-Low/4-High 

Following  the  experiment,  each  technician  filled  out  a 
questionnaire  with  18  questions  to  determine  what  each 
subject  thought  about  the  cursor-keys  and  joystick  and  to 
allow  them  to  recommend  other  access  methods  to  explore  in 
future  research  (See  Appendix  F) .  Then,  each  subject  was 
interviewed  to  gain  their  observational  insight  and  to 
determine  their  satisfaction  level  while  using  each  access 
device.  During  the  interview,  respondents  were  given  an 
opportunity  to  critique  the  access  methods  and  the  design  of 
the  experiment  (See  Appendix  G) . 

Subjects 

Thirty-two  aircraft  related  maintenance  technicians 
were  randomly  selected  from  the  4950th  Test  Wing  at  Wright- 
Patterson  AFB,  OH  and  divided  in  two  separate  groups  based 
on  their  experience  using  computers.  All  selections  were 
made  on  a  random,  independent  basis.  The  technicians  were 
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categorized  as  experienced  if  they  had  over  1  year  of 
experience  using  computers  and  as  an  apprentice  if  they  had 
1  year  or  less  experience. 

Data  Collection 

Total  task  and  sub-task  completion  times  were  recorded 
by  an  observer  as  the  technicians  performed  each  routine. 
Each  routine  included  three  individual  sub-tasks,  which  were 
timed  and  recorded  to  allow  further  analysis  and  comparison. 
Time  measurement  began  when  the  technician  started  each  sub¬ 
task  and  stopped  when  it  was  completed.  Total  task  times 
were  computed  by  summing  the  sub-task  times  and  dividing  by 
60  to  convert  them  to  minutes.  The  same  observer  was  used 
throughout  this  portion  of  the  experiment  to  reduce  bias. 

Each  respondent  was  interviewed  by  the  second 
researcher  to  investigate  information  not  readily  apparent 
from  the  experiment  and  the  questionnaire  answers.  To 
reduce  bias,  the  questions  listed  in  Appendix  G  were  used  to 
maintain  consistency.  Answers  to  the  interview  questions 
were  used  to  help  support  the  experimental  results. 

Scope  Limitations 

The  experiment  was  originally  supposed  to  use  the  PMA 
only,  however,  a  schedule  conflict  caused  the  researchers  to 
employ  a  combination  UNISYS  personal  computer  (PC)  and  PMA 
keyboard.  This  change  led  to  the  requirement  for  the 
technicians  to  look  at  both  the  keyboard  and  the  PC  screen; 
the  PMA  would  have  precluded  this  extra  necessity. 
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The  research  was  conducted  with  the  most  recent  PMA 
available  which  contained  four  cursor-keys  and  a  thvimb  knob 
device,  instead  of  a  typical  joystick.  The  uncharacteristic 
nature  of  the  operation  of  this  device  may  adversely  affect 
the  experimental  results  for  the  joystick. 

The  presentation  system  used  by  the  Armstrong 
Laboratory  utilized  an  unusual  cross-stitch  method  which 
connected  a  horizontal  and  vertical  scheme  to  move  between 
different  selectable  regions  on  the  screen.  This  system  is 
also  somewhat  uncommon,  especially  to  experienced  computer 
users.  Obviously,  this  liability  could  also  be  detrimental 
to  the  results  obtained  from  the  experiment. 

Even  though  this  experiment  was  conducted  to  determine 
the  best  access  method  to  employ  in  an  outdoor,  flight  line 
environment,  a  controlled  area  was  used  because  of  equipment 
and  power  requirements. 

Experimental  Hypotheses 

The  investigative  questions  outlined  in  Chapter  One  of 
this  thesis  served  as  the  foundation  for  this  research. 

1.  The  joystick  device  will  access  technical 
information  faster  than  the  four-cursor  keys. 

2.  Technicians  using  the  joystick  device  will  have 
greater  satisfaction  than  those  using  the  four-cursor  keys. 

3.  Experienced  technicians  will  require  less  access 
time  than  inrxperienced  technicians. 
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4.  A  single  access  method,  the  joystick,  will  work 
best  for  all  types  of  tasks? 

The  prime  concern  of  this  research  was  to  determine  which 
access  method  was  the  fastest  and  had  the  greatest  amount  of 
user  satisfaction. 

Method  of  Analysis 

An  ANOVA  (Analysis  of  Variance)  and  Linear  Regression 
were  used  to  evaluate  the  research  results.  An  overall  F- 
test  with  a  0.05  significance  level  was  used  to  test  for 
statistical  significance. 
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ly.  Experimental  R§gults  ^  Analysis 


As  mentioned  in  previous  chapters,  32  maintenance 
technicians  with  varying  levels  of  computer  and  maintenance 
experience  participated  in  this  experiment.  Data  in  several 
forms  were  analyzed  using  Statistix,  version  3.5,  to 
determine  the  level  of  significance  of  the  hypotheses.  The 
methods  used  included  the  Analysis  of  Variance  (ANOVA) , 
Linear  Regression  (LR) ,  graphs  and  histograms.  ANOVA  and  LR 
results  are  summarized  in  this  chapter  while  Appendixes  H 
and  I  contain  graphs  and  histograms.  In  addition  to  the 
time  interval  data  presented  below,  other  information 
provided  by  the  subjects  from  interviews  or  questionnaires 
aided  in  analyzing  two  hypotheses. 

The  first  evaluation  was  conducted  on  the  raw  test  data 
which  included  computer  processing  time  for  display  of 
information  to  the  screen  and  user  response  time.  Each  task 
was  divided  into  three  phases.  Table  5  lists  the  times  for 
each  phase  in  seconds  and  total  task  time  in  minutes.  The 
table  also  shows  technician  computer  experience  in  years  and 
their  military  rank/civilian  grade  and  specialty  code.  The 
data  is  arranged  into  four  groups  based  upon  which  cursor 
control  device  and  task  were  used  first.  The  boxes  with  two 
extra  vertical  lines  designate  which  access  method  was  used 
first. 
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TABLE  5 


CURSOR-KEY  AND  JOYSTICK  TASK  COMPLETION  DATA 
(INCLUDES  COMPUTER  RESPONSE  TIMES) 


NM 

BR 

FIRST 

CPTR 

EXP. 

(yr) 

MIL 

RNK 

E-? 

SPEC. 

CODE 

JOYSTICK 

CURSOR-KEYS 

TSK 

MTD 

Phsel 

(sec) 

Phse2 

(sec) 

Phse3 

(sec) 

TOTL 

mins 

TOTL 

mins 

Phsel 

(sec) 

Phse2 

(sec) 

Phse3 

(sec) 

1 

Joy 

2.25 

5 

45770 

223 

299 

la 

11.3 

Wl 

210 

300 

215 

ff 

ff 

4 

2 

45532 

174 

246 

m  ^ 

9.3 

166 

261 

186 

ft 

ff 

6 

5 

457X2 

183 

270 

mm 

10.1 

fra 

189 

300 

194 

n 

ff 

7 

3 

316X3 

174 

261 

133 

9.5 

10.4 

166 

273 

183 

5 

ff 

ff 

0 

CIV 

454X1 

201 

255 

142 

10.0 

11.1 

185 

270 

214 

6 

ff 

ff 

1 

5 

458X2 

208 

299 

169 

11.3 

12.7 

227 

298 

234 

7 

n 

ff 

1 

5 

45471 

192 

267 

133 

9.9 

11.5 

211 

275 

205 

8 

ff 

ff 

1 

6 

45475 

221 

303 

175 

11.6 

13.4 

227 

317 

257 

9 

1 

Key 

1.08 

5 

45471 

197 

286 

260 

12.4 

11.1 

207 

285 

Bl 

EQ 

ff 

ff 

4 

7 

458X2 

178 

278 

220 

11.3 

10.0 

200 

269 

■1c  m 

11 

ft 

ft 

2 

2 

31633 

171 

262 

185 

10.3 

9.9 

185 

258 

mm 

12 

ff 

ft 

1.17 

CIV 

216 

313 

259 

13.1 

12.1 

251 

296 

177 

13 

H 

ft 

0.83 

CIV 

45451 

200 

294 

252 

12.4 

11.1 

236 

283 

147 

14 

ft 

ff 

1 

5 

45871 

226 

292 

227 

12.4 

10.8 

214 

287 

149 

15 

ff 

ft 

1 

5 

45770 

196 

324 

244 

12.7 

10.6 

209 

258 

168 

16 

ft 

ft 

1 

CIV 

220 

284 

271 

12.9 

13.0 

250 

338 

193 

17 

2 

Joy 

3 

45471 

248 

321 

256 

13.8 

11.3 

229 

288 

162 

18 

ff 

ff 

9 

45451 

217 

278 

210 

11.8 

9.0 

152 

255 

130 

19 

mm 

ff 

2.67 

45873 

201 

298 

225 

12.1 

10.5 

215 

263 

153 

20 

ft 

2.5 

45873 

183 

261 

201 

10.8 

8.8 

157 

253 

115 

21 

B 

ft 

1 

45451 

203 

269 

221 

11.6 

8.6 

155 

237 

123 

22 

B 

ff 

1 

5 

45451 

213 

295 

255 

12.7 

10.3 

192 

275 

152 

23 

B 

ft 

0.33 

5 

45451 

255 

320 

251 

13.8 

11.8 

226 

306 

175 

24 

m 

n 

0 

5 

45471 

231 

305 

222 

12.6 

10.1 

176 

299 

133 

25 

2 

Key 

1.75 

3 

31653 

202 

297 

147 

10.8 

12.4 

225 

296 

222 

26 

ff 

ft 

2 

CIV 

207 

294 

150 

10.8 

13.8 

223 

300 

305 

27 

mm 

ff 

1.5 

5 

45770 

222 

319 

200 

12.4 

14.4 

268 

342 

264 

28 

B 

ff 

4 

3 

31653 

195 

284 

144 

10.4 

12.6 

206 

297 

254 

29 

B 

ff 

0.83 

CIV 

45470 

219 

333 

174 

12.1 

15.4 

270 

318 

331 

30 

B 

ff 

0.75 

5 

31653 

200 

268 

127 

9.9 

11.6 

195 

275 

224 

31 

B 

ff 

1 

5 

45770 

239 

335 

200 

12.9 

14.3 

272 

302 

283 

32 

■ 

ff 

0 

4 

45451 

227 

296 

155 

11.3 

14.8 

289 

324 

276 

NOTES:  Air  Force  Specialty  Codes  listed  without  shredouts 

Extra  vertical  lines  designate  which  access  method  was  used  first 
Each  group  has  four  experienced  (one  year  or  more)  and  four 
inexperienced  subjects 
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The  following  table  shows  how  the  joystick  raw  data 
task  time  compares  against  the  raw  data  for  the  cursor-keys. 
Table  6  shows  that  the  raw  data  completion  times  for  the  two 
complete  tasks  differed  by  1.74  minutes  or  nearly  1  minute 
and  45  seconds.  More  importantly,  the  overall  average  time 
for  the  two  access  methods  (underlined)  were  only  0.03 
minutes  (less  than  2  seconds)  apart. 

TABLE  6 

CURSOR-KEY  AND  JOYSTICK  TASK  COMPLETION  DATA  SUMMARY 
(INCLUDES  COMPUTER  RESPONSE  TIMES) 


GRP 

FIRST 

MTHD 

MEAN/ 

STANDRO 

DEVIATN 

JOYSTICK 

CURSBR  KEYS 

Phasel 

(secs) 

Phaae2 

(sees) 

Phaee3 

(secs) 

TOTAL 

(Min) 

TOTAL 

(Min) 

Phasel 

(secs) 

Phase2 

(secs) 

Phase! 

(secs) 

B 

Joy 

Mean 
Std  Dev 

197 

18.3 

275 

20.8 

150 

15.0 

10.37 

0.849 

11.59 

1.006 

198 

23.2 

287 

18.2 

211 

23.5 

2 

Key 

Mean 
Std  Dev 

201 

18.3 

292 

18.2 

240 

26.2 

12.20 

0.886 

11.07 

0.977 

219 

22.5 

284 

24.1 

161 

18.8 

3 

Joy 

Mean 
Std  Dev 

219 

22.9 

293 

20.9 

230 

19.9 

12.37 

0.989 

10.04 

1.116 

188 

30.3 

272 

22.7 

143 

19.4 

B 

Key 

Mean 
Std  Dev 

214 

14.3 

303 

22.2 

162 

25.0 

11.32 

0.971 

13.67 

1.246 

244 

33.0 

307 

19.2 

270 

35.1 

To¬ 

tal 

N/A 

Mean 
Std  Dev 

208 

20.8 

291 

23.0 

196 

45.5 

11»56 

1.222 

i;»59 

1.713 

212 

35.0 

287 

24.6 

196 

55.3 

OTHER  AVERAGE  TOTAL  TIME  STATISTICS 


TASK  NUMBER  ONE 

10.72  Nlns 

1st  TASK  FINISHED 

11.87  Mins 

TASK  NUMBER  TWO 

12.46  Nins 

2nd  TASK  FINISHED 

11.29  Mins 

As  can  be  seen  from  the  total  average  phase  times 
listed  in  the  summary  table,  the  three  phases  are  not  equal. 
Both  tasks  were  similar  enough  that  the  combination  of 
phases  one  and  two  of  both  tasks  entailed  approximately  the 


same  movements.  The  same  was  true  for  phase  three.  A  short 
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list  of  the  features  of  each  task  phase  is  listed  below  (See 
Appendixes  B  and  C) : 

Phases  One  and  Two: 

1)  Navigate  a  block  diagram  and  select  specified 
regions  in  the  illustration. 

2)  Use  pull-down  menu  to  display  calculator  and  choose 
specified  numbers  to  calculate  the  product  of  two  numbers. 

3)  Scroll  through  lists  and  highlight  specific  "hot 
spots"  to  answer  system  queries. 

4)  Move  to  a  particular  block  in  a  diagram  and  enter  a 
built-in-test  code  using  the  nvimeric  keypads. 

5)  Relocate  the  cursor  to  a  specified  item  in  a  list 
and  choose  it  using  the  "Select"  key. 

6)  Use  a  function  key  to  access  a  list  of  the  times 
that  each  task  movement  was  performed  and  record  the  time 
for  a  specific  movement  task. 

Phase  Three: 

7)  Utilize  the  "Part  Info"  function  key  to  produce  a 
parts  list  and  then  find  and  select  a  part  to  be  ordered. 

8)  Press  a  function  key  to  display  a  cursor  activated 
on-screen  keyboard  and  then  move  the  cursor  to  the  required 
letters  to  define  the  quantity  and  packaging  for  the  part. 

9)  Push  the  "next"  key  to  scroll  through  the  part 
ordering  routine  of  the  system. 

The  only  elements  which  did  not  require  intensive  cursor 
movement  were  numbers  six  and  nine. 
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Tables  7-10  display  information  comparing  individual 


phase  times  for  each  access  method,  as  well  as  total 
completion  times  for  the  tasks  used  in  the  experiment.  To 
be  precise,  table  7  presents  the  data  related  to  the  "total 
times"  to  complete  the  tasks  using  the  two  access  methods. 

TABLE  7 

ONE  WAY  ANOVA  TEST  FOR  BOTH  TASKS  -  Total  all  Phase  Times 
(INCLUDES  COMPUTER  RESPONSE  TIMES) 


SOURCE 

DF 

SUM  OF  SQUARES 

MEAN  SQUARE 

F 

P 

BETWEEN 

1 

0.01351 

0.01351 

0.01 

0.9390 

WITHIN 

62 

141.7 

2.286 

TOTAL 

63 

141.7 

VARIABLE 

MEAN 

STD  DEV 

CHI  SQ 

P 

JOYSTICK 

11.56 

1.241 

CURSOR-KEYS 

11.59 

1.741 

BARTLETT'S  TEST  FOR  EQUAL  VARIANCES 

3.43 

0.0639 

COCHRAN'S 

Q 

0.6631 

LARGEST  VAR  /  SMALLEST  VAR 

1.968 

COMPONENT 

OF  VARIANCE  FOR  BETWEEN  GROUPS 

-0.07101 

According  to  the  "p-value"  listed  in  table  7,  there  is 
a  mere  6 . 1  percent  chance  that  these  access  method  times  are 
different.  The  Bartlett's  and  Cochran's  Q  tests  and  the 
F-test  value  of  0.01  help  support  this  claim. 

The  next  three  tables  present  the  ANOVA  information  for 
each  of  the  phases  of  both  tasks  used  to  test  both  cursor 
control  devices. 
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TABLE  8 


ONE  WAY  ANOVA  TEST  FOR  BOTH  TASKS  -  Phase  One  Only 
(INCLUDES  COMPUTER  RESPONSE  TIMES) 


TABLE  9 

ONE  WAY  ANOVA  TEST  FOR  BOTH  TASKS  -  Phase  Two  Only 
(INCLUDES  COMPUTER  RESPONSE  TIMES) 


SOURCE 

OF 

SUM  OF  SQUARES 

MEAN  SQUARE 

F 

P 

BETWEEN 

1 

182.3 

182.3 

0.31 

0.5784 

WITHIN 

62 

3.621E-»-04 

584.0 

TOTAL 

63 

3.639E-»-04 

VARIABLE 

MEAN 

STD  DEV 

CHI  SQ 

P 

JOYSTICK 

290.8 

23.32 

CTOSOR-KEYS 

287.4 

24.98 

BARTLETT'S  TEST  FOR  EQUAL  VARIANCES 

0.14 

0.7045 

COCHRAN'S 

Q 

0.5343 

LARGEST  VAR  /  SMALLEST  VAR 

1.147 

COMPONENT 

OF  VARIANCE  FOR  BETWEEN  GROUPS 

-12.56 
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TABLE  10 


ONE  WAY  ANOVA  TEST  FOR  BOTH  TASKS  -  Phase  Three  Only 
(INCLUDES  COMPUTER  RESPONSE  TIMES) 


SOURCE 

OF 

SUM  OF  SQUARES 

MEAN  SQUARE 

F 

P 

BETWEEN 

WITHIN 

TOTAL 

1 

62 

63 

8.266 

1.640B-*-05 

1. 6408+05 

8.266 

2646 

0.00 

0.9556 

VARIABLE 

MEAN 

STD  DEV 

CHI  SQ 

P 

JOYSTICK 

CURSOR-KEYS 

195.5 

196.2 

46.20 

56.19 

BARTLETT'S  TEST  FOR  EQUAL  VARIANCES 

1.16 

0.2809 

COCHRAN'S  Q 

LARGEST  VAR  /  SMALLEST  VAR 

COMPONENT  OF  VARIANCE  FOR  BETWEEN  GROUPS 

0.5967 

1.479 

-82.42 

Tables  7-10  show  that  the  phase  times  are  well  linked 
to  each  other,  validating  the  results  presented  in  table  6, 
that  the  two  access  methods  perform  the  same. 

To  obtain  more  meaningful  information,  the  researchers 
performed  both  tasks  to  determine  the  amount  of  time  it  took 
for  the  computer  to  respond  with  a  new  screen  after  the 
previous  one  was  completed.  These  computer  times  were  added 
together  within  each  phase  and  then  subtracted  from  the 
previous  totals  so  the  aggregate  time  (listed  in  minutes) 
indicated  only  the  human  response  times  and  no  computer 
processing  times.  The  next  two  tables  exhibit  the  operator 
response  time  data  for  the  phases,  the  whole  task,  and  the 
statistics  data.  In  fact,  table  11  shows  the  same 
information  as  table  5,  minus  the  computer  response  times. 
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TABLE  11 


CURSOR-KEY  AND  JOYSTICK  TASK  COMPLETION  DATA 
(COMPUTER  RESPONSE  TIMES  SUBTRACTED) 


NM 

BR 

FIRST 

CPTR 

CPTR 

EXP. 

(yr) 

MIL 

MIL 

RNK 

E-? 

SPEC. 

CODE 

JOYSTICK 

CURSOR-KEYS 

TSK 

MTD 

Phsel 

(sec) 

Phse2 

(sec) 

Phse3 

(sec) 

TOTL 

mins 

TOTL 

mins 

Phsel 

(sec) 

Phse2 

(sec) 

Phse3 

(sec) 

1 

Joy 

2.25 

5 

45770 

178 

180 

138 

8.27 

7.10 

146 

117 

163 

f« 

tt 

4 

2 

45532 

129 

127 

120 

6.27 

5.23 

102 

78 

134 

tl 

It 

6 

5 

451X2 

138 

151 

133 

7.03 

6.40 

125 

117 

142 

M 

tl 

7 

3 

316X3 

129 

142 

114 

6.42 

5.38 

102 

90 

131 

5 

ft 

tl 

0 

CIV 

454X1 

156 

136 

123 

6.92 

6.17 

121 

87 

162 

6 

ft 

tt 

1 

5 

458X2 

163 

180 

150 

8.22 

7.67 

163 

115 

182 

7 

tl 

tt 

1 

5 

45471 

147 

148 

114 

6.82 

6.53 

147 

92 

153 

8 

tl 

M 

1 

6 

45475 

176 

184 

156 

8.60 

8.37 

163 

134 

205 

9 

1 

Key 

1.08 

5 

45471 

133 

103 

208 

7.40 

8.05 

162 

166 

155 

10 

tt 

It 

4 

7 

458X2 

114 

95 

168 

6.28 

6.97 

155 

150 

113 

11 

tt 

tt 

2 

2 

31633 

107 

79 

133 

5.32 

6.82 

140 

139 

130 

12 

tt 

H 

1.17 

CIV 

152 

130 

207 

8.15 

9.02 

206 

177 

158 

13 

tl 

tt 

0.83 

CIV 

45451 

136 

111 

200 

7.45 

8.05 

191 

164 

128 

tt 

tt 

1 

5 

458X1 

162 

109 

175 

7.43 

7.78 

169 

168 

130 

tt 

tl 

1 

5 

45770 

132 

141 

192 

7.75 

7.53 

194 

139 

149 

16 

tt 

tt 

1 

CIV 

156 

101 

219 

7.93 

9.97 

205 

219 

174 

17 

2 

Joy 

3 

5 

45471 

■  [  ■ 

138 

204 

8.77 

8.27 

184 

169 

143 

18 

tt 

tl 

9 

4 

45451 

^pTcB 

95 

158 

6.77 

5.90 

107 

136 

111 

19 

mm 

tt 

2.67 

5 

45873 

115 

173 

7.08 

7.47 

170 

144 

134 

20 

tt 

2.5 

5 

45873 

119 

78 

149 

5.77 

5.70 

112 

134 

96 

21 

D 

It 

1 

4 

45451 

139 

86 

169 

6.57 

5.53 

110 

118 

104 

22 

tt 

1 

5 

45451 

149 

112 

203 

7.73 

7.27 

147 

156 

133 

23 

B 

N 

0.33 

5 

45451 

191 

137 

199 

8.78 

8.73 

181 

187 

156 

24 

ft 

tt 

0 

5 

45471 

167 

122 

170 

7.65 

7.08 

131 

180 

114 

25 

2 

Key 

1.75 

3 

31653 

157 

178 

128 

7.72 

7.40 

161 

113 

170 

26 

tt 

tt 

2 

CIV 

162 

175 

131 

7.80 

8.82 

159 

117 

253 

27 

mm 

tt 

1.5 

3 

45773 

177 

20C 

381 

9.30 

9.58 

204 

159 

212 

28 

B 

tt 

4 

3 

31653 

150 

165 

123 

7.30 

7.63 

142 

114 

202 

29 

B 

tt 

HU 

45470 

174 

214 

155 

9.05 

10.3 

206 

135 

279 

30 

B 

N 

5 

31653 

155 

149 

108 

6.87 

6.58 

131 

92 

172 

31 

B 

tt 

5 

45770 

194 

216 

181 

9.85 

9.30 

208 

119 

231 

32 

o 

tt 

4 

45451 

182 

177 

136 

8.25 

9.83 

225 

141 

224 

NOTES:  First  task  (tsk)  and  method  (mtd)  shown  for  all  four  groups 

Extra  vertical  lines  designate  which  access  method  was  used  first 
Each  group  has  four  experienced  (one  year  or  more)  and  four 
inexperienced  subjects 
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TABLE  12 


CURSOR-KEY  AND  JOYSTICK  TASK  COMPLETION  DATA  SUMMARY 
(COMPUTER  RESPONSE  TIMES  SUBTRACTED) 


GRP 

FIRST 

MTHD 

MEAN/ 

STANDRD 

DEVIATN 

JOYSTICK 

CURSER  KEYS 

Phasel 

(sees) 

Phase2 

(secs) 

Pha8e3 

(secs) 

TOTAL 

(Min) 

TOTAL 

(Min) 

Phasel 

(secs) 

Pha8e2 

(secs) 

Phase3 

(secs) 

■ 

Joy 

Mean 
Std  Dev 

1S2 

18.3 

156 

20.8 

131 

15.0 

7.317 

0.849 

6.606 

1.006 

134 

23.2 

104 

18.2 

159 

23.5 

2 

Key 

Mean 
Std  Dev 

137 

18.3 

109 

18.2 

188 

26.2 

7.215 

0.886 

8.023 

0.977 

174 

22.5 

165 

24.1 

142 

18.8 

3 

Joy 

Mean 
Std  Dev 

155 

22.9 

110 

20.9 

178 

19.9 

7.390 

0.989 

6.994 

1.116 

143 

30.3 

153 

22.7 

124 

19.4 

■ 

Key 

Mean 
Std  Dev 

169 

14.3 

184 

22.2 

143 

25.2 

8.267 

0.975 

8.685 

1.246 

180 

33.0 

124 

19.2 

218 

35.1 

To¬ 

tal 

N/A 

Mean 
Std  Dev 

153 

21.9 

140 

38.0 

160 

32.2 

7.547 

1.017 

7.577 

1.367 

157 

33.9 

136 

32.1 

161 

43.3 

OTHER  AVERAGE  TOTAL  TIME  STATISTICS 


1st  TASK  FINISHED 


7,85  Mins 


2nd  TASK  FINISHED 


7.27  Mins 


Although  the  researchers  subtracted  the  computer 
processing  times  from  the  raw  data,  the  times  for  the 
joystick  and  cursor-keys  remained  comparable.  After 
comparing  each  phase  of  both  tasks,  the  widest  separation 
noted  was  only  4  seconds.  This  disparity  occurred  twice  in 
phases  1  and  2.  It's  interesting  to  note  that  the  joystick 
was  faster  than  the  cursor-keys  in  phase  one,  but  was  slower 
in  phase  two.  The  researchers  could  not  conclude  the  cause 
of  this  variance.  Also,  it's  remarkable  that  the  learning 
curve  of  0.58  minutes  or  about  35  seconds  (see  underlined 
numbers)  is  only  7.7  percent  of  the  total  time.  Although 
not  a  hypothesis,  the  researchers  expected  more  variation  in 
the  access  times  due  to  a  cognitive  learning  curve. 
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since  the  overall  average  times  for  the  two  tasks  were 
relatively  close  (roughly  10  seconds  out  of  seven  and  one- 
half  minutes) ,  a  one-on-one  comparison  of  the  times  for  each 


subject  was  conducted.  Table  13  lists  the  task  times  for 
each  technician,  whichever  method  or  order  was  utilized. 
Table  14,  on  page  53,  displays  the  statistical  information 
furnished  for  these  pairs. 

TABLE  13 

TASK  il  VERSUS  TASK  #2  COMPARISON  DATA  -  Total  Times 
(COMPUTER  RESPONSE  TIMES  SUBTRACTED) 


TASK  ONE  FIRST 

TASK  TWO  FIRST 

SBJT 

TASK 

TASK 

SBJT 

TASK 

TASK 

NMBR 

#1 

#2 

NMBR 

#1 

*2 

8.27 

7.10 

8.27 

8.77 

6.27 

5.23 

Wm 

5.90 

6.77 

7.03 

6.40 

7.47 

7.08 

6.42 

5.38 

20 

5.70 

5.77 

5 

6.92 

6.17 

21 

5.53 

6.57 

6 

8.22 

7.67 

22 

7.27 

7.73 

7 

6.82 

6.53 

23 

8.73 

8.78 

8 

8.60 

8.37 

24 

7.08 

7.65 

9 

8.05 

7.40 

25 

7.72 

7.40 

10 

6.97 

6.28 

26 

7.80 

8.82 

11 

6.82 

5.32 

27 

9.30 

9.58 

12 

9.02 

8.15 

28 

7.30 

7.63 

13 

8.05 

7.45 

29 

9.05 

10.33 

14 

7.78 

7.43 

30 

6.87 

6.58 

15 

7.53 

7.75 

31 

9.85 

9.30 

16 

9.97 

7.93 

32 

8.25 

9.83 

NOTE:  Subjects  1-16  completed  Task  #1  first 
and  the  others  started  with  Task  #2 
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TABLE  14 


ONE  WAY  ANOVA  TEST  FOR  TASK  #1  VERSUS  TASK  #2 
(COMPUTER  RESPONSE  TIMES  SUBTRACTED) 


SOURCE 

DF 

SUM  OF  SQUARES 

MEAN  SQUARE 

F 

P 

BETWEEN 

1 

0.5041 

0.5041 

0.34 

0.5629 

WITHIN 

62 

92.38 

1.490 

TOTAL 

63 

92.89 

VARIABLE 

MEAN 

STD  DEV 

CHI  SQ 

P 

TASK  NUMBER  1 

7.651 

1.128 

TASK  NUMBER  2 

7.473 

1.307 

BARTLETT'S  TEST  FOR  EQUAL  VARIANCES 

0.66 

0.4177 

COCHRAN'S 

Q 

0.5730 

LARGEST  VAR  /  SMALLEST  VAR 

1. 

342 

COMPONENT 

OF  VARIANCE  FOR  BETWEEN  GROUPS 

-0.03081 

Statistically,  the  two  tasks  were  comparable,  as 
verified  by  the  relatively  large  p-value  (0.5629)  and  the 
closeness  of  the  two  standard  deviations.  One  theory  could 
stipulate  that  the  order  of  the  tasks  must  be  the  major 
cause  of  the  disparity  of  the  values  even  though  the 
learning  curve  was  considered  negligible. 

Hypothesis  Number  X 

This  hypothesis  speculates  that  the  joystick  device 
will  access  technical  information  faster  than  the  four- 
cursor  keys.  Tables  15-18  present  the  access  time  data  for 
the  whole  task  and  each  phase  for  the  technicians,  allowing 
a  true  comparison  of  only  the  human  response  times. 


TABLE  15 


ONE  WAY  ANOVA  TEST  FOR  BOTH  TASKS  -  Total  All 

Phase  Times 

(COMPUTER  RESPONSE  TIMES  SUBTRACTED) 


OF  SUM  OF  SQUARFS 


0.01381 

92.87 

92.89 


7.548 

7.577 


SOURCE 

DF 

BETWEEN 

1 

WITHIN 

62 

TOTAL 

63 

VARIABLE 

JOYSTICK 

CURSOR-KEYS 

SQUARE 

0. 

1 

01381 

.498 

STD  DEV 

1 

1 

.033 

.389 

BARTLETT'S  TEST  FOR  EQUAL  VARIANCES 


COCHRAN'S  Q 

LARGEST  VAR  /  SMALLEST  VAR 

COMPONENT  OF  VARIANCE  FOR  BETWEEN  GROUPS 


0.6439 

1.808 

-0.04638 


TABLE  16 

ONE  WAY  ANOVA  TEST  FOR  BOTH  TASKS  -  Phase  One  Only 
(COMPUTER  RESPONSE  TIMES  SUBTRACTED) 


SOURCE 

DF 

SUN  OF  SQUARES 

MEAN  SQUARE 

BETWEEN 

1 

310.6 

310.6 

WITHIN 

62 

5.218E+04 

841.5 

TOTAL 

63 

5.249E-^04 

VARIABLE 

MEAN 

STD  DEV 

JOYSTICK 

153.1 

22.29 

CURSOR-KEYS 

157.5 

_ 1 

34.44 

BARTLETT'S  TEST  FOR  EQUAL  VARIANCES 

COCHRAN'S 

Q 

LARGEST  VAR  /  SMALLEST  VAR 

COMPONENT 

OF  VARIANCE  FOR  BETWEEN  GROUPS 

0.7047 

2.386 

-16.59 
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TABLE  17 

ONE  WAY  ANOVA  TEST  FOR  BOTH  TASKS  -  Phase  Two  Only 
(COMPUTER  RESPONSE  TIMES  SUBTRACTED) 


SOURCE 

DF 

SUM  OF  SQUARES 

MEAN  SQUARE 

F 

P 

BETWEEN 

1 

182.3 

182.3 

0.14 

0.7070 

WITHIN 

62 

7.922E+04 

1278 

TOTAL 

63 

7.940E+04 

VARIABLE 

MEAN 

STD  DEV 

CHI  SQ 

P 

JOYSTICK 

139.8 

38.59 

CURSOR-KEYS 

136.4 

32.65 

BARTLETT'S  TEST  FOR  EQUAL  VARIANCES 

0.85 

0.3565 

COCHRAN'S 

Q 

0.5829 

LARGEST  VAR  /  SMALLEST  VAR 

1. 

397 

COMPONENT 

OF  VARIANCE  FOR  BETWEEN  GROUPS 

-34.23 

TABLE  18 

ONE  WAY  ANOVA  TEST  FOR  BOTH  TASKS  -  Phase  Three  Only 
(COMPUTER  RESPONSE  TIMES  SUBTRACTED) 


SOURCE 

DF 

SUM  OF  SQUARES 

MEAN  SQUARE 

F 

P 

BETWEEN 

1 

9.766 

9.766 

0.01 

0.9360 

WITHIN 

62 

9.326E+04 

1504 

TOTAL 

63 

9.327E+04 

VARIABLE 

MEAN 

STD  DEV 

CHI  SQ 

P 

JOYSTICK 

159.9 

32.81 

OmSOR-KEYS 

160.7 

43.95 

BARTLETT'S  TEST  FOR  EQUAL  VARIANCES 

2.57 

0.1087 

COCHRAN'S 

Q 

0.6422 

LARGEST  VAR  /  SMALLEST  VAR 

1. 

795 

OMPONENT 

OF  VARIANCE  FOR  BETWEEN  GROUPS 

-46.70 
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Although  table  15  shows  that  the  joystick  finished  the 
tasks  first,  the  extremely  small  difference  in  the  two 
completion  times  and  the  high  p-value  combine  to  validate 
that  there  is  no  statistical  difference  between  the  access 
tools.  The  phase  charts  establish  that  the  joystick 
performed  better  (faster)  during  two  of  the  three  sections, 
numbers  one  and  three.  However,  the  p-values  were  very  high 
indicating  that  the  variation  between  the  task  times  were 
insignificant.  Therefore,  the  first  hypothesis,  that  "the 
joystick  will  access  information  faster"  was  rejected  at  the 
0.05  significance  level. 

Hypothesis  Number  Z 

The  second  supposition  maintains  that  "technicians 
using  the  joystick  device  will  have  greater  satisfaction 
than  those  utilizing  the  four-cursor  keys."  The  information 
provided  by  the  questionnaire  and  personal  interview  was 
employed  to  evaluate  this  premise  (See  Appendixes  H  and  I) . 
Table  19,  a  summary  of  the  data  listed  in  Appendix  H,  shows 
that  the  cursor-keys  were  more  popular  with  the  technicians 
by  an  average  23  to  9  margin.  In  addition,  24  of  the  32 
subjects  chose  the  cursor-keys  as  the  device  they  would  use 
in  response  to  question  2  of  the  personal  interview  (See 
Appendix  I) ,  "which  computer  access  method  did  you  prefer?" 
The  unconventional  shape  and  operation  of  the  joystick  may 
be  somewhat  responsible  for  this  lopsided  count. 
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TABLE  19 


SUMMARY  OF  QUESTIONNAIRE  INFORMATION 


RESEARCH  QUESTIONS 

AGREE 

DISAGREE 

UNO. 

FOUR-CURSOR  KEYS 

■ 

were  best  to  access/select  information: 

■ 

in  the  calculator. 

23 

7 

in  all  block  diagrams. 

23 

6 

in  the  parts  lists. 

23 

5 

■1 

in  the  menu. 

22 

8 

H 

caused  fewer  accessing/selection  errors. 

25 

0 

B 

were  preferred  to  access/select  all  tasks. 

23 

3 

6 

THUHBKNOB 

■1 

was  best  to  access/select  information: 

B 

in  the  calculator. 

8 

9 

H 

in  all  block  diagrams. 

8 

11 

ill 

in  the  parts  lists. 

10 

9 

13 

in  the  menu. 

9 

11 

12 

caused  fewer  accessing/selection  errors. 

9 

6 

17 

was  preferred  to  access/select  all  tasks. 

B 

8 

17 

Neither  method  worked  best  to  access  and 

5 

12 

15 

select  information  for  all  tasks. 

Accessing  requested  information  was  difficult 

14 

6 

B 

due  to  the  way  the  cursor  moved  across  the  screen. 

B 

I  would  recommend  another  access  method 

B 

8 

B 

I  would  recommend  using  a  light  pen. 

12 

B 

I  would  recommend  using  a  mouse. 

B 

8 

13 

I  would  recommend  a  different  access  method. 

B 

6 

20 

Thus,  the  null  hypothesis,  "the  technicians  will  prefer 
the  joystick  over  the  cursor-keys,"  is  not  proven  since  the 
cursor  keys  were  selected  by  a  majority  of  the  personnel. 


Hypothesis  Number  3 


This  hypothesis  states  that  "experienced  technicians 
will  require  less  time  to  access  computer  generated 
information  than  technicians  with  one  year  or  less  computer 
experience."  The  researchers  sorted  the  task  completion 
times  based  upon  the  computer  experience  level  of  each 
technician.  Then  a  linear  regression  line  testing  the 
completion  times  for  both  tasks  against  each  technician's 
computer  experience  level  was  created.  Table  20  documents 
the  results  of  this  comparison  for  the  combined  times  of  the 
two  tasks  using  both  access  methods.  The  p-value  of  0.0084 
reveals  that  there  is  a  99.2  percent  chance  that  the  level 
of  computer  experience  is  significant  for  this  test  case. 

TABLE  20 

LINEAR  REGRESSION  OF  COMPUTER  EXPERIENCE  VS  TOTAL  TIME  - 

Both  Tasks 

(COMPUTER  RESPONSE  TIMES  SUBTRACTED) 


SOURCE 

DF 

SUM  OF  SQUARES 

MEAN  SQUARE 

F 

P 

REGRESSION 

RESIDUAL 

TOTAL 

1 

62 

63 

27.825 

104.99 

132.82 

27.825 

3.4997 

7.95 

0.0084 

VARIABLES 

COEFFICIENT 

STD  ERROR 

T 

P 

CONSTANT 

TOTAL 

6.3151 

-0.04124 

_ 

2.2366 

0.014626 

3.72 

-2.82 

0.0008 

0.0084 

R  SQUARED 

ADJUSTED  R  SQUARED 

0.2095 

0.1832 

■ 

MEAN  SQUARE  ERROR 
STANDARD  DEVIATION 

3.500 

1.871 
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TABLE  21 


LINEAR  REGRESSION  OF  COMPUTER  EXPERIENCE  VS  TIME  FOR 

KEYBOARD 

(COMPUTER  RESPONSE  TIMES  SUBTRACTED) 


SOURCE 

DF 

SUM  OF  SQUARES 

MEAN  SQUARE 

F 

P 

REGRESSION 

RESIDUAL 

TOTAL 

1 

30 

31 

19.662 

113.15 

132.82 

19.662 

3.7718 

5.21 

0.0297 

VARIABLES 

COEFFICIENT 

STD  ERROR 

T 

P 

CONSTANT 

JOYSTICK 

7.8972 

-0.77103 

2.5719 

0.033771 

3.07 

-2.28 

0.0045 

0.0297 

R  SQUARED 

ADJUSTED  R  SQUARED 

0.1480 

0.1196 

■ 

MEAN  SQUARE  ERROR 
STANDARD  DEVIATION 

3.772 

1.942 

TABLE  22 

LINEAR  REGRESSION  OF  COMPUTER  EXPERIENCE  VS  TIME  FOR 

CURSOR-KEYS 

(COMPUTER  RESPONSE  TIMES  SUBTRACTED) 


SOURCE 

DF 

SUN  OF  SQUARES 

MEAN  SQUARE 

F 

P 

REGRESSION 

1 

29.458 

29.458 

8.55 

0.0065 

RESIDUAL 

30 

103.36 

3.4453 

TOTAL 

31 

132.82 

VARIABLES 

COEFFICIENT 

STD  ERROR 

P 

CONSTANT 

7.3958 

1.8480 

4.00 

0.0004 

TOTAL 

-( 

). 70187 

0.24003 

-2.92 

0.0065 

R  SQUARED 

0.2218 

MEAN  SQUARE  ERROR 

3.445 

ADJUSTED  R  SQUARED 

0.1959 

STANDARD  DEVIATION 

1.856 

After  comparing  the  p-values  (0.0297  vs  0.0065)  listed 
in  tables  21  and  22,  it  is  obvious  that  the  operation  of  the 
cursor  keys  was  most  affected  by  the  level  of  computer 
experience.  One  potential  reason  for  this  difference  is 
that  most  computer  users  must  become  familiar  with  the 
operation  of  cursor-keys  to  perform  the  majority  of  the 
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functions  on  a  personal  computer.  Even  the  high  p-value  for 
the  cursor-keys  transforms  to  a  97  percent  chance  that  each 
subject's  experience  level  is  significant.  Therefore,  the 
third  hypothesis,  "a  more  experienced  technician  will 
complete  each  task  faster,"  is  not  rejected  at  a  95  percent 
confidence  level. 

Hypothesis  Number  4 

The  final  hypothesis,  "the  joystick,  will  work  best  for 
all  types  of  tasks,"  was  tested  using  questions  2  and  3  of 
the  personal  interview  (See  Appendix  I)  when  the  researcher 
asked,  "which  access  method  did  you  prefer"  and  "did  your 
choice  work  best  in  all  tasks?"  Of  the  30  technicians  who 
had  a  preference,  six  (only  20  percent)  chose  the  joystick 
device.  However,  all  six  subjects  who  picked  the  joystick 
preferred  it  for  every  task.  Of  the  24  subjects  who  chose 
the  cursor-keys,  only  one  did  not  want  to  use  it  all  the 
time.  Since  the  20  percent  joystick  selection  rate  is  not 
statistically  meaningful  at  a  0.05  significance  level,  this 
hypothesis  is  rejected.  Nevertheless,  Appendix  I  indicates 
that  29  of  the  30  test  subjects  would  have  used  their  access 
choice  for  every  task  presented  in  this  experiment.  Thus, 
it  is  possible  that  a  "typical"  joystick  device  might  be 
endorsed  by  a  significant  number  of  maintenance  technicians 
during  a  future  experiment,  should  one  ever  be  performed. 
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V.  Discussions.  Conclusions,  and  Recommendations 


Discussion  of  Quantitative  Results 

Data  collected  during  this  experiment  indicated  that 
there  are  no  statistical  differences  in  access  times  using 
either  the  joystick  device  or  the  four-cursor  key 
configuration.  As  a  result  of  these  findings  it  was 
concluded  that  the  joystick  device  does  not  access 
information  faster  and  with  greater  user  satisfaction  than 
users  of  the  four-button  cursor  keys.  However,  two  factors 
surface  as  possible  obstacles  to  the  hypothesized  outcome. 
The  joystick  device  chosen  by  Armstrong  Laboratory  did  not 
conform  to  a  true  joystick  device  as  described  in  Chapter 
II.  Also,  the  software  program,  Smalltalk  V,  did  not  allow 
the  full  range  of  movement  (eight  directions)  normally 
associated  with  a  joystick  access  device.  Subsequent 
sections  will  discuss  these  impediments  in  greater  detail. 

Task  Environment .  This  experiment  took  place  in  a 
small  branch  office  of  the  Aerospace  Ground  Equipment  hangar 
of  the  4950th  Test  Wing  and  was  not  designed  to  simulate 
flight  line  conditions.  Neither  the  computer  unit  nor  the 
technicians  were  exposed  to  loud  noise,  aircraft  exhaust 
fumes,  dirt  and  grime,  and  the  mobility  synonymous  with 
tecnnicians  working  in  a  flight  line  environment. 
Furthermore,  technicians  were  required  to  accessed 
information  from  two  maintenance  tasks  for  an  aircraft  that 
they  were  not  familiar,  the  F-18.  Since  no  Air  Force 
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specialty  was  exempted  from  this  experiment,  the  researchers 
felt  that  a  unfamiliar  task  would  ultimately  test  the  access 
devices,  not  the  prior  maintenance  experience  of  the 
technician.  Technician  movement  was  also  limited.  Subjects 
sat  in  a  chair  positioned  directly  in  front  of  an  office 
desk  that  supported  the  UNISYS  PC  computer  with  the  Portable 
Maintenance  Aid  (PMA)  keyboard  attached.  Technicians  were 
then  instructed  by  a  researcher  to  access  information  from 
the  maintenance  routine  (researchers  desired  to  measure 
access  time  not  reading  ability  and  response  time) .  Each 
task  was  broken  down  into  phases  and  once  a  phase  was 
completed,  the  researcher  documented  the  response  time. 

This  experiment  did  not  test  the  agility  of  the  PMA  or  the 
ability  of  the  technician  to  use  each  access  device  while 
maneuvering  around  equipment  configurations  in  an  aircraft. 

F-18  Maintenance  Task.  All  test  subjects  performed  the 
same  two  maintenance  F-18  test  routines,  the  only  difference 
being  the  access  device  used  to  perform  the  routine  and  the 
sequence  in  which  the  tasks  were  performed.  The  F-18  test 
routines  had  been  used  on  numerous  demonstration  and  test 
studies  performed  by  Armstrong  Laboratory  and  provided  the 
researchers  with  a  variety  of  accessible  data.  Individual 
routines  were  designed  so  that  every  technician  would  have 
to  scroll  across,  or  up  and  down  a  computer  screen  to  access 
different  types  of  information.  During  the  course  of  each 
routine  a  technician  accessed  information  necessary  to  order 
parts,  chose  items  from  a  menu,  documented  information  on 
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supply  forms,  performed  calculations  using  the  calculator 
function,  and  chose  replacement  parts  by  scrolling  a  parts 
list.  It  was  observed  that  novice  technicians  had  a 
tendency  to  keep  both  hands  on  the  PMA  keyboard.  The  left 
hand  was  positioned  to  enable  quick  depression  of  the  NEXT 
key  and  the  FUNCTION  keys  that  were  located  on  the  lower 
left  side  of  the  PMA.  The  right  hand  was  positioned  over 
the  lower  right  portion  of  the  PMA  to  operate  the  joystick 
(thumb  knob)  and  the  four-button  cursor  keys.  The 
experienced  technicians  (those  having  more  than  one  year's 
experience  using  computers)  had  a  tendency  to  relax  the  hand 
which  was  not  absolutely  necessary  to  perform  the  task  and 
move  it  into  position  as  required.  It  was  also  noted  that 
experienced  technicians  would  use  their  left  hand  to  perform 
some  functions  on  the  right  hand  side  of  the  keyboard. 
Consequently,  the  experienced  technicians  appeared  more 
confident  using  the  computer  than  the  unexperienced  ones. 
Although  this  situation  was  noted,  neither  technique 
appeared  to  hamper  the  response  time  of  a  technician  nor  did 
it  result  in  a  large  number  of  selection  errors. 

Hardware  Configuration.  As  noted  in  Chapter  III,  the 
hardware  configuration  consisted  of  a  PMA  keyboard  connected 
to  the  keyboard  jack  of  a  UNISYS  PC  computer.  The  PMA 
keyboard  was  constructed  so  that  the  cursor  keys  and  the 
thumb  knob  were  located  on  the  lower  right  hand  side. 
Consequently,  both  video  access  devices  were  operated  with 
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the  subject's  right  hand  and  technicians  were  forced  to 
alternate  between  their  right  index  finger  and  their  right 
thumb  to  make  selections.  Many  technicians  complained  that 
the  thumb  knob  was  hard  to  depress  and  unresponsive  while  a 
few  complained  that  it  was  too  sensitive.  It  soon  become 
obvious  to  the  researchers  that  the  majority  of  the 
technicians  preferred  the  cursor  keys  over  the  thiamb  knob. 

As  indicated  in  Chapter  IV,  statistically,  there  was  no 
difference  in  access  times  between  the  cursor  keys  and  the 
thumb  knob.  The  researchers  hypothesize  that  this  situation 
occurred  because  the  thumb  knob  tested  did  not  function,  nor 
was  it  configured  like  a  true  joystick  device  as  described 
in  Chapter  II.  The  thumb  knob  was  constructed  to  be 
pressure  sensitive  as  well  as  directionally  sensitive.  Each 
technician  was  required  to  place  downward  pressure  on  the 
thumb  knob  to  activate  it  while  at  the  same  time  moving  the 
thumb  knob  in  an  up/down,  left/right  direction.  If  both 
techniques  were  not  properly  employed,  the  curse  would 
either  not  move  or  move  in  an  undesired  direction.  This 
caused  technicians  some  frustration.  Also,  although  the 
thumb  knob  could  be  moved  in  a  diagonal  direction,  the 
cursor  would  not  respond  in  a  like  manner  because  of 
software  programming  limitations.  Both  researchers 
attribute  some  of  the  dissatisfaction  of  the  thumb  knob  by 
the  users  and  the  lack  of  statistical  difference  of  access 
times  to  the  construction  and  operation  of  the  thumb  knob 
device  and  associated  software. 
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Software  Configuration.  Smalltalk  V,  the  application 
software  program,  that  allowed  display  and  maneuverability 
within  the  F-18  demonstration  routine  had  limitations  that 
restricted  the  operation  of  the  thumb  knob  (joystick) . 

First,  the  thumb  knob  restricted  cursor  movement  to  four, 
instead  of  eight,  directions.  Normally  a  joystick  device 
with  supporting  hardware  will  accommodate  cursor  movement  in 
eight  directions,  cursor  up,  cursor  diagonally  to  the  right, 
cursor  right,  cursor  diagonally  downward,  cursor  down, 
cursor  diagonally  to  the  left,  cursor  left,  and  cursor 
diagonally  upward.  The  thumb  knob  used  for  this  experiment 
could  only  move  the  cursor  in  an  up/down,  or  left/right 
direction;  diagonal  movement  was  not  possible. 

Both  access  devices  were  limited  by  the  software  as  to 
selection  of  objects  on  the  screen.  Smalltalk  V  would 
sequence  every  object  in  relationship  to  its  numerical  order 
(cursor  would  jump  from  item  1  then  to  item  2  and  so  forth) 
regardless  of  the  cursor  key  selected  or  the  direction 
chosen  by  the  thumb  knob.  Consequently,  technicians  would 
depress  either  a  cursor  key  or  the  thumb  knob  expecting  it 
to  move  in  one  direction  when  it  actually  moved  in  another. 
Although  this  situation  was  overcome  by  the  technician 
during  the  cognitive  learning  curve,  it  solicited  negative 
comments  from  the  majority  of  technicians  performing  this 
experiment. 

The  last  software  limitation  involved  the  turbo  action 
of  the  thumb  knob.  If  the  thumb  knob  was  held  in  a 


65 


depressed  mode,  the  cursor  would  build  up  momentum  that  was 
difficult  to  react  to.  In  some  instances,  technicians 
overshot  their  intended  choices  when  the  cursor  failed  to 
stop  after  the  thumb  knob  was  released.  Some  of  the  more 
experienced  technicians  adapted  to  this  peculiar  operating 
feature  while  the  novice  technicians  became  somewhat 
unnerved.  The  researchers  believe  that  these  limitations 
may  have  significantly  affected  the  results  of  this 
experiment  since  both  devices  tested  statistically 
insignificant . 

Discussion  of  Qualitative  Results 

Both  the  questionnaire  and  the  personal  interview  data 
suggests  that  the  majority  of  technicians  preferred  the 
four-button  cursor  key  access  method  over  the  thumb  knob 
access  method.  And  a  few  technicians  felt  that  neither 
method  would  be  suited  for  the  flight  line.  A  few 
technicians  suggested  alternate  methods,  touch  sensitive 
screen,  pen  light,  and  the  mouse,  but  voiced  concern  whether 
these  access  devices  could  survive  the  hazards  of  the  flight 
line. 

All  but  one  technician  felt  that  computerized  technical 
orders  were  a  definite  improvement  over  paper-based 
technical  orders.  This  lone  technician  felt  that 
developmental  cost  of  an  automated  technical  order 
outweighed  its  usefulness.  The  remainder  of  technicians 
cited  ease  of  use,  rapid  and  immediate  access  to  technical 
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information,  elimination  of  missing  or  soiled  pages  in  a 
paper  technical  order,  etc.,  as  the  main  advantages  of  using 
automated  technical  orders. 

Several  technicians  were  concerned  whether  the  PMA 
could  withstand  the  oil  and  solvent  contaminants  used  on  the 
flight  line.  One  hydraulics  technician  informed  the 
researchers  of  a  hydraulic  fluid,  Skydrol,  that  dissolves 
plastic  and  rubber  components.  This  information  was 
compiled  and  given  to  Armstrong  Laboratory  for  further 
research. 

Conclusions 

The  results  of  this  experiment  suggest  that  the  four- 
cursor  key  access  method  is  the  method  of  preferred  choice 
for  maintenance  technicians  tested.  Although  all 
technicians  performed  equally  well  accessing  information 
using  either  access  method  (statistically  there  was  no 
difference  in  access  times) ,  qualitative  data  clearly 
identifies  the  cursor  keys  as  the  most  preferred  access 
device.  As  stated  before,  the  researchers  believe  that  the 
joystick  device  employed  on  the  PMA  resulted  in  this  outcome 
as  this  device  was  difficult  for  technicians  to  operate. 

The  most  encouraging  outcome  of  this  experiment  was  the 
overwhelming  acceptance  of  automated  technical  orders.  All 
technician  felt  that  an  automated  TO  system  would  be  an 
improvement  over  the  present  paper-based  system.  Although 
technicians  voiced  concerned  about  the  PMAs  survival  on  the 
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flight  line  and  its  reliability  and  repair  cost,  all  the 
technicians  felt  that  they  would  be  able  to  complete  their 
jobs  faster  using  an  automated  technical  order. 


Recommendations  £s£  Follow-on  Experiments 

Accessing  available  technical  information  is  an 
important  part  of  any  technician's  job  and  paper-based 
technical  orders  have  become  too  bulky  and  cumbersome  to  use 
on  the  flight  line.  Automated  technical  orders  provide  the 
next  logical  progression  to  alleviate  this  problem. 

However,  automated  technical  orders  can  also  become 
frustrating  for  technicians  to  use  if  they  are  unable  to 
retrieve  stored  information  using  an  access  device  that  they 
prefer  and  are  accustom  to  using.  Cursor  keys  offer 
technicians  a  default  alternative  but  there  are  better  and 
faster  methods  available  and  new  access  methods  are 
constantly  being  developed.  The  researchers  believe  that 
the  joystick  device  is  still  a  viable  alternative. 
Furthermore  we  recommend  that  another  test  be  performed 
comparing  the  cursor  key  configuration  against  the  joystick 
device,  but  that  another  joystick  device  resembling  those 
employed  on  computer  games  be  used  versus  the  thumb  knob 
design  used  for  this  test.  A  joystick  device  that  included 
a  selection  button  on  its  top  was  suggested  by  several 
technicians  during  the  interview  segment  of  this  experiment. 
Other  recommended  access  devices  were  also  received  and  a 
list  was  provided  to  Armstrong  Laboratories. 


Another  recommended  Improvement  would  be  software 
programming.  A  joystick  device  works  best  if  a  technician 
is  given  the  ability  to  move  in  all  directions  not  just 
four.  Some  consideration  should  be  given  in  changing  the 
present  software  programming  package  to  one  that  fully 
employs  the  eight  directional  operating  characteristics  of  a 
joystick  device. 

This  study  did  not  require  technicians  to  move  about  a 
computer  screen  to  select  an  electronic  component  or  trace 
signal  paths  within  a  schematic  diagram.  The  researchers 
believe  that  a  test  that  incorporated  this  aspect  of  a 
technician's  job  would  be  extremely  helpful  in  evaluating 
the  usefulness  of  an  access  device.  Tracing  a  signal  path 
would  force  a  technician  to  move  in  all  directions  and  would 
be  a  truer  test  of  user  satisfaction  and  system  agility. 

The  final  suggestion  involves  evaluation  of  access 
devices  as  the  technician  actually  performs  a  maintenance 
routine.  The  researd/'.ers  believe  that  movement  of  a 
technician  in  the  performance  of  a  maintenance  or  job  task 
will  significantly  affect  which  access  method  the  technician 
will  choose  to  use.  Once  further  tests  narrow  down  the 
access  choices,  that  choice  should  be  tested  under  flight 
line  conditions  to  determine  its  suitability  before  being 
employed  in  the  operational  model  of  the  PMA. 
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Appendix  A 
Experimental  Plan 
I.  Description  of  Evaluation 


Purpose . 

The  purpose  of  this  research  is  to  determine  the 
quickest  and  most  satisfying  way  to  access  maintenance 
information  from  a  computerized  data  base,  a  joystick  or 
four-cursor  keys. 


Sardware. 

A  combined  UNISYS  Personal  Computer  (PC)  and  Integrated 
Maintenance  Information  System  (IMIS)  Portable  Maintenance 
Aid  (PMA)  prototype  keyboard  was  used  in  alj.  phases  of  the 
study  to  increase  reliability. 


Software. 

The  software  used  during  this  investigation  includes 
MSDOS  5.0  and  the  Armstrong  Laboratory  developed  authoring 
and  presentation  system  called  the  F-18  Presentation  System 
which  allows  maintenance  information  about  the  F-18  aircraft 
to  be  digitized.  The  F-18  Presentation  System  was  composed 
on  version  1.1  of  Smalltalk  V,  a  program  which  supports  the 
formulation  and  display  of  graphical  information. 


Software  System  Limitations. 

Smalltalk  V  utilizes  a  cross  stitch  cursor  movement 
procedure  which  combines  horizontal  and  vertical  selection 
techniques  to  designate  the  next  region  to  be  chosen.  While 
this  process  allows  a  user  to  select  any  displayed  region, 
it  is  somewhat  unconventional,  which  may  affect  how  the  test 
subjects  respond  to  researcher  instructions. 


Subjects. 

There  will  be  a  total  of  32  maintenance  technicians  (16 
experienced  and  16  non-experienced)  from  several  shops 
within  the  4950th  Test  Wing  located  at  Wright-Patterson  Air 
Force  Base  OH  participating  in  this  experiment.  These 
technicians  will  be  divided  into  four  categories  to  ensure 
each  task  and  access  method  combination  is  used  first  by  an 
equal  number  of  subjects  to  minimize  bias  effects. 
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Tasks. 


Each  maintenance  technician  will  complete  two  simulated 
maintenance  tasks  which  will  require  them  to  find  and  access 
technical  information  from  several  different  screens  and 
configurations.  The  tasks  will  simulate  the  procedures  to 
administer  a  F-18  Nose  Wheel  Steering  Built-In-Test  (BIT) 
and  a  Main  BIT  and  should  be  comparable  based  on  difficulty 
level  and  completion  time.  One  task  will  be  accomplished 
using  one  access  method  and  the  other  method  will  be  used 
for  the  second  task.  The  order  for  task  completion  and 
which  access  method  to  employ  will  be  randomly  deterr.ined 
based  upon  which  of  the  four  groups  each  technician  as 
assigned  to  (See  Subjects  paragraph  above) .  Table  23 
summarizes  the  method  and  task  order  assigned  to  each  group. 

TABLE  23 


ACCESS  METHOD  AND  GROUP  ORDER  USED  FOR  SIMULATED  TASKS 


GROUP  NUM 

FIRST  TASK 

FIRST  METHOD 

EXPERIENCE 

GROUP  1 

Task  1 

Joystick 

4 -Low/ 4 -High 

GROUP  2 

Task  2 

Joystick 

4 -Low/ 4 -High 

GROUP  3 

Task  1 

Cursor-Key 

4 -Low/ 4 -High 

GROUP  4 

Task  2 

Cursor-Key 

4 -Low/ 4 -High 

Both  tasks  used  during  this  experiment  were  validated  during 
a  pilot  study  using  students  from  the  Air  Force  Institute  of 
Technology  stationed  at  Wright-Patterson  Air  Force  Base  OH. 
None  of  the  subjects  had  any  Air  Force  maintenance 
experience,  but  they  all  had  substantial  experience  with 
personal  computer  applications.  No  one  used  during  the 
pilot  study  will  be  used  during  the  actual  experiment. 

The  task  displays  are  representative  of  what  a 
technician  would  need  to  actually  perform  maintenance  on  the 
built-in-test  portions  of  a  F-18  aircraft. 


Data  Collected. 

Demographic  data  will  be  collected  using  the  Background 
Survey  (Appendix  D) .  Some  of  the  information  provided  by 
this  form  will  be  used  to  determine  the  experience  level  of 
each  technician  and  to  assign  them  to  a  group.  Also,  during 
the  experiment,  notes  and  observations,  including  the 
technique  utilized  by  each  experimenter  to  maneuver  about 
the  PMA  screen  and  intermediate,  error,  and  total  times  will 
be  recorded.  A  questionnaire  will  be  used  to  ascertain  each 
user's  evaluation  of  both  access  methods  (Appendix  F) .  Then 
a  researcher  will  conduct  an  interview  using  Appendix  G  to 
obtain  comments  on  both  access  methods,  including  any 
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preferences.  The  following  information  will  be  collected 
during  the  performance  of  each  task: 

Total  time  to  finish  the  complete  task. 

Time  to  complete  phase  one 

Time  to  complete  phase  two 

Time  to  complete  phase  three 

Error  times  to  subtract  from  each  phase 


Hypotheses . 

The  four  research  questions  and  their  associated  hypotheses 
are  listed  in  Chapter  I  or  Chapter  III  respectively. 


Controls. 

The  following  actions  will  be  executed  throughout  the 
experiment  to  control  experimental  variation  due  to  computer 
software  programs. 

1.  The  response  times  for  both  tasks  will  be  compiled 
and  subtracted  to  ensure  the  sub-task  and  total  task  times 
are  representative  of  the  actual  task  performance.  The 
response  times  measured  represent  the  times  required  to 
retrieve  text  screens,  graphics  screens,  and  any  lists 
required  to  discharge  the  tasks.  Statistical  analysis  will 
be  performed  both  before  and  after  the  response  times  are 
subtracted  to  examine  any  differences. 

2.  All  32  subjects  will  be  randomly  selected  and 
assigned  to  each  experimental  group  within  their  experience 
category.  The  designated  group  will  dictate  the  set  of 
conditions  they  will  undergo  during  the  experiment. 

3.  All  data  collection  will  be  performed  in  the  same 
location  at  the  4950th  Test  Wing  by  one  researcher.  The 
other  researcher  will  conduct  the  personal  interviews  and 
review  the  questionnaire  in  a  separate  private  location  at 
the  Test  Wing. 

4.  Both  researchers  will  conduct  their  activities 
throughout  the  experiment  to  minimize  any  bias. 


Performance  Measurement. 

The  following  guidelines  will  be  used  to  control  the 
data  collection  period  and  ensure  that  performance  times 
represent  the  actual  time  required  to  complete  the  simulated 
tasks. 
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1.  If  a  technician  is  experiencing  difficulty  in 
accessing  the  information  because  of  program  or  training 
problems,  the  experimenter  will  aid  the  technician  as 
required  to  assure  the  resulting  times  are  representative  of 
their  real  capability. 

2.  If  a  subject  has  not  completed  a  task  within  20 
minutes,  that  task  will  be  aborted.  In  accordance  with  the 
schedule,  each  technician  will  be  allotted  a  total  of  50 
minutes  to  complete  both  tasks  and  move  on  to  the 
questionnaire  and  interview  section  of  the  experiment. 


II.  Procedures  to  Conduct  Experiment 


Sequence  of  Events . 

1.  Completion  of  Background  Survey  (Appendix  D) . 

2.  Introduce  experiment  and  demonstrate  experimental 
device  to  group. 

3.  Random  assignment  of  subjects  to  experimental 
groups . 

4.  Short  "hands-on”  training  before  executing  tasks. 

5.  Perform  both  tasks  using  access  methods  dictated  by 
the  assigned  group. 

6.  Fill  out  questionnaire. 

7.  Complete  personal  interview  conducted  by  second 
researcher. 


Introduction . 

Each  technician  will  receive  a  short  description  of  the 
purpose  and  instructions  for  this  experiment.  The  subjects 
will  also  be  reminded  that  participation  in  this  research  is 
completely  voluntary  and  that  their  names  will  not  be  used 
anywhere  in  the  research  report.  Task  performance  and 
answers  to  the  questionnaire  or  personal  interview  will  not 
affect  their  job  performance  ratings  in  any  way.  See 
appendix  G  for  the  questions  the  experimenter  will  ask  each 
subject. 


Training- 

Each  technician  will  receive  "hands-on"  training  with 
the  PMA  immediately  prior  to  use  to  ensure  familiarity  with 
its  operation.  During  this  short  training  period,  the 
researcher  will  be  available  to  answer  any  questions  that 
may  arise.  Once  the  technician  is  comfortable  with  the 
system  to  be  used,  the  experiment  will  begin. 
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Debriefing. 


After  each  subject  executes  both  tasks,  a  questionnaire 
and  personal  interview  will  be  administered.  Before  the  end 
of  each  session,  a  researcher  will  remind  the  subjects  not 
to  talk  to  anyone  else  about  the  experiment  until  it's 
completed.  Finally,  technicians  will  be  briefed  on  how 
their  experimental  data  will  be  used. 
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Appendix  B 
Task  1  Listing 

TEST  OF  NOSE  WHEEL  STEERING  BUILT-IN-TEST 


The  following  list  was  used  by  the  researchers  to  lead 
the  maintenance  technicians  through  the  task  to  test  the 
operation  of  the  two  cursor  control  devices. 

LOG-ON 

Log  onto  the  PMA;  enter  "3”  for  "TECHNICIAN  #1*8 
NUMBER",  then  press  <NEXT> 

Press  <NEXT>  to  choose  "SESSION  1" 


BEGIN  PHASE  ONE  (Start  Timing) 

Press  <NEXT>  one*  to  reach  the  '*Integrated  Flight 
Control  System"  screen 

Highlight  "#7.  Mission  Computer",  press  <SELECT>,  then 
highlight  and  <SELECT>  "#7.  Elect  System" 

Now  press  <CANCEL>  button  twice  and  choose  IFCS,  then 
press  <NEXT>  twice 

At  "FCS  NWS  BIT  Procedure",  choose  and  <SELECT>  "Prelim 
with  Hydro",  then  press  <NEXT> 

Choose  "No"  to  1553  Hookup,  press  <SELECT>,  then  press 
<NEXT> 

Press  <NEXT>  five  times  to  reach  "procedure  #7" 

Press  <MENU>,  then  <SELECT>  "7.  Utilities"  followed  by 
"Calculator"  to  display  the  Calculator 

Highlight  and  <SELECT>  numbers  3,9,&2,  then  <SELECT> 
"*",  followed  by  1,8,&3;  check  the  answer:  71,736 

Press  <CANCEL>  button  three  times  back  to  "procedure 
#7" 

Press  <NEXT>  six  times  (past  notes  boxes)  to  reach 

"procedure  #8" 

Press  <NEXT>  once  to  reach  "procedure  #9" 
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END  PHASE  ONE /BEGIN  PHASE  TWO 

Choose  INTEST/DEGD  for  FSCA  only,  press  <SELECT>,  then 
press  <NEXT> 

Press  <NEXT>  twice  to  reach  Process  BLZN  results,  then 
press  <NEXT> 

Choose  "No”  to  1553  Hookup,  press  <SELECT>,  press 
<NEXT> 

Press  <NEXT>  twice  to  reach  "procedure  #4" 

Press  LOG  FILE  Soft  Key  <F4>;  Scroll  down  the  Log  File 
and  record  time  for  "Start  Task:  Read  BLIN",  then  press 
<F1> 

Press  <NEXT>  three  times  to  reach  "BLIN  CODE  READING" 
screen  with  Note  Box,  then  press  <NEXT>  again 

Highlight  a  "CHANNEL  2"  box,  enter  "4744",  press  <NEXT> 

Press  <NEXT>  twice  to  reach  the  "Block  Diagram"  (WAIT) 

Highlight  and  <SELECT>  "11,  IPCS",  then  highlight  and 
<SELECT>  "3.  Nose  Wheel  Steering" 

Press  RANKED  ACTIONS  Soft  Key  <F1> 

Highlight  and  select  option  #4,  "NWS  Selector. . .1903H", 
press  <SELECT>,  then  press  <NEXT>  twice 

Choose  and  <SELECT>  all  the  "Required  Conditions",  then 
press  <NEXT>  three  times 

Highlight  and  <SELECT>  "No",  at  84P-G036  PIN  prompt, 
then  press  <NEXT>  twice 


END  PHASE  TWO/BEGIN  PHASE  THREE 

At  "Block  Diagram",  press  RANKED  ACTIONS  Soft  Key  <F1> 

Highlight  and  <SELECT>  option  #5,  "Repair 
Wiring. . .Relay"  then  press  <NEXT> 

Press  PART  INFO  Soft  Key  <F1>  to  produce  parts  list 

Highlight  a  "#7  Circuit  Breaker/Relay  Panel  Assembly", 
press  PART  INFO  Soft  Key  <F1> 

At  "Relay  Electromagnetic,  2PDT"  display,  press 
<SELECT>,  then  press  <NEXT> 
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Press  <F3>  807T  XBY  to  use  the  Keyboard  to  enter  a 
Quantity  of  "lOS",  then  <SELECT>  "SEMD  VALUE" 

Now  choose  "Unit  of  Issue"  and  press  <F3>  SOFT  KEY  to 
use  the  Keyboard  to  enter  "BOX",  then  <SELECT>  "SEND 
VALUE" 

Press  ORDER  key  <F1>  to  "Order"  this  part 

Press  <NEXT>  five  times  to  reach  the  "FCCA  Input 
Conditions**  screen 

Press  <MENU>,  then  <SELECT>  "1.  File"  followed  by  "SET 
UP"  to  reset  the  computer 


PHASE  THREE/L06-0FF 

Press  <NEXT>  to  "MAINTENANCE  CLOSEOUT  INFORMATION", 
then  press  <NEXT>  again 

At  "Inspector  code",  enter  then  press  <NEXT> 
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T?^sK  2.  Listing 
TEST  OF  MAIN  BUILT-IN-TEST 

The  following  list  was  used  by  the  researchers  to  lead 
the  maintenance  technicians  through  the  task  to  test  the 
operation  of  the  two  cursor  control  devices. 

L06-0N 

Log  onto  the  PMA;  enter  “a"  for  "TECHNICIAN  #1*8 
NUMBER",  then  press  <NEXT> 

Scroll  and  <SELECT>  "TRAINING  SESSION  2",  press  <NEXT> 


BEGIN  PHASE  ONE  (Start  Timing) 

At  "Session  Information"  press  FAULT  VERIFY  Soft  Key 
<F4> 

At  "Fault  Verification"  screen,  press  <NEXT>  to  choose 
MAIN  BIT,  then  press  <NEXT> 

At  "FCS  Main  BIT  Procedure",  choose  and  <SELECT> 

"PRELIM  WITH  HYDRO",  press  <NEXT> 

Choose  "No"  to  1553  Hookup,  press  <SELECT>,  then  press 
<NEXT> 

Press  <NEXT>  four  times  to  reach  "procedure  #6" 

Then  choose  DEGD  for  both  FSCA  and  FSCB  by  pressing 
<SELECT>,  then  press  <NEXT> 

Choose  and  <SELECT>  all  the  "Required  Conditions" 

Press  <NEXT>  once  to  "BLIN  Code  Reading",  then  press 
<NEXT>  again 

Press  <SELECT>  to  choose  "Electrical  Power",  then  press 
<NEXT> 

Choose  "No"  to  1553  Hookup,  press  <SELECT>,  then  press 
<NEXT> 
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END  PHASE  OME/BEOIM  PHASE  TWO 


Press  <NEXT>  twice  to  reach  "procedure  #4" 

Press  <MENU>,  then  highlight  "Utilities"  followed  by 
"Celculetor"  to  display  the  Calculator 

Highlight  and  <SELECT>  numbers  2,7, &1,  then  <SELECT> 
"*",  followed  by  7,2,&9;  check  the  answer:  197,559 

Press  <CANCEL>  button  three  times  back  to  "procedure 
#4" 

Press  <NEXT>  four  times  to  reach  "BLIM  Code  Reeding" 
procedure  check 

Highlight  "CHANNEL  2",  enter  "4263",  press  <NEXT>  twice 

Highlight  "Electrical  Power",  press  <SELECT> 

Press  <NEXT>  once  to  reach  the  "Block  Diagram"  WAIT! ! 

Highlight  "#15.  Aileron",  press  <SELECT>,  then 
highlight  and  <SELECT>  "#18.  Hyd  Sys" 

Now  press  <CANCEL>  twice  and  highlight  "Elect  Sys", 
press  RANKED  ACTIONS  Soft  Key  <F1> 

Press  <NEXT>  to  choose  "FCCA  Continuity  Check  1902B" 

Choose  and  <SELECT>  all  the  "Required  Conditions",  then 
press  <NEXT>  twice 

Highlight  and  <SELECT>  "No"  at  "Continuity"  prompt, 
press  <NEXT> 

Choose  and  <SELECT>  all  the  "Required  Conditions" 

Press  <NEXT>  once  to  reach  the  "Block  Diagram"  WAIT! 1 


END  PHASE  TWO/ BEGIN  PHASE  THREE 

At  "Block  Diagram",  press  RANKED  ACTIONS  Soft  Key  <F1> 

Highlight  and  <SELECT>  option  #2,  "Repair  Wiring 
between. . .PCCA",  then  press  <NEXT> 

Press  PART  INFO  Soft  Key  <F1>  to  produce  a  Parts  List 

Press  <SCROLL>  Key  and  cursor  key  to  scroll  UP  the  list 
to  the  top,  then  choose  a  "Roll-Pitch** Yaw  Computer 
(FCCB)  and  press  the  PART  INFO  Soft  Key  <F1> 
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Press  <SELECT>  to  choose  a  "Coaputer,  Roll-Pitch-Yaw” , 
press  <NEXT> 

Press  <F3>  SOFT  KEY  to  use  Keyboard  to  enter  a  Quantity 
of  "aoi",  then  <SELECT>  "SEND  VALUE" 

Now  choose  "Unit  of  Issue"  and  press  <F3>  SOFT  KEY  to 
use  the  Keyboard  to  enter  "CRATE",  then  <SELECT>  "SEND 
VALUE" 

Press  ORDER  Soft  Key  <F1>  to  "Order"  this  part 

Press  <NEXT>  five  times  to  reach  the  "Repair  wiring" 
screen 

Press  <MENU>,  then  <SELECT>  "1.  File"  followed  by  "SET¬ 
UP"  to  reset  the  computer 


END  PHASE  THREE/LOG-OFF 

Press  <NEXT>  to  "MAINTENANCE  CLOSEOUT  INFORMATION", 
then  press  <NEXT>  again 

At  "Inspector  Code",  enter  "99",  then  press  <NEXT> 


Appendix  D 
Background  Survey 

EXP? 

Subject  Identification  Number  _  GROUP  # 

1.  Check  one:  Military  _  Civilian  _ 


2. 

Total  Active  Military 

service 

in  Yrs 

and  months. 

3. 

Current  pay  grade: 

4. 

Current  job  specialty 

or  rating: 

5. 

Prior  Work  Experience 

(crew  chief. 

hydraulic 

technician. 

etc. ) : 

(1) 

(2) 

(3) 

Career  specialty: 

Location: 

Years  experience: 

Inclusive  dates  (m/y) 

_/. 

(4) 

(5) 

(6) 

Occupation  specialty: 

Location: 

Years  experience: 

Inclusive  dates  (m/y) 

_/. 

_J_/_ 

6. 

Education/ Formal  training: 

High  grade  completed 

(high  school= 

12, 

college  =  16) :  _ 

Department  of  Defense 

Schools  completed: 

Courses : 

Years  experience: 

Inclusive  date 

_/. 

7. 

Computer/Electronic  Experience: 

Formalized  Training: 

_  Yes 

No 

Self-taught: 

_  Yes 

No 

Computer  Model/Type:  _  Exp  (yr/mths) 
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Appendix  1 
Observation  Form 

Task  Number:  _  Subject  ID: 


Phase 

1 

Times 

Start : 

Compl: 

Error : 

- 

Phase 

2 

Times 

Start : 

Compl: 

Error : 

Phase 

3 

Times 

Start: 

Compl: 

Error : 

t 

Phase 

4 

Times 

Start : 

Compl: 

Error : 

Time  Task  terminated  (if  unsuccessfully  completed):  _ 

Number  of  steps  successfully  completed:  _ 

Number  of  steps  not  completed:  _ 

The  sequence  number  of  step  not  completed:  _ 

Reason  for  not  completing:  _ 

xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx 

Problems/Notes : 
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ID  #. 


User  Evaluation  Questionnaire 

Please  answer  the  following  questions  based  on  you 
participation  in  the  computer  interface  evaluation.  After 
reading  the  question  place  a  mark  in  the  appropriate  box. 
The  scales  are  as  follows: 


1 .  Strongly  Agree 

2 .  Agree 

3 .  Undecided 

4 .  Disagree 

5.  Strongly  Disagree 


I.  Questions  on  the  button  cluster  access  method 


1.  Four-cursor  keys  were  superior  when 
accessing  and  selecting  information  in  the 
calculator. 

2 .  Four-cursor  keys  were  superior  when 
accessing  and  selecting  information  in  all 
block  diagrams. 

3 .  Four-cursor  keys  were  superior  when 
accessing  and  selecting  information  in  the 
parts  lists 

4 .  Four-cursor  keys  were  superior  when 
accessing  and  selecting  information  in  the 
menu. 

5.  Fewer  accessing  and  selection  errors  were 
made  using  the  button  cluster  keys  versus 
the  thumb  knob. 

6.  Four-cursor  keys  are  the  preferred  method 
for  accessing  and  selecting  all  tasks 
performed  during  this  evaluation. 
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II.  Questions  on  the  thumb  knob  access  method. 


7.  The  thumb  knob  is  superior  when  accessing 
and  selecting  information  in  the  calculator. 

8.  The  thumb  knob  is  superior  when  accessing 
and  selecting  information  in  block  diagrams. 

9.  The  thumb  knob  is  superior  when  accessing 
and  selecting  information  in  parts  lists. 

10.  The  thumb  knob  is  superior  when  accessing 
and  selecting  informa+-ion  in  the  menu. 

11.  Fewer  accessing  and  section  errors  were 
made  using  the  thumb  knob  versus  the 
cursor  key  method. 

12.  The  thumb  knob  is  the  preferred  method  for 
accessing  and  selecting  all  tasks  performed 
during  this  evaluation. 


I 


ill.  Other  Influencing  Factors 


13.  Neither  the  cursor  keys  nor  the  thumb  knob 
access  method  worked  best  in  accessing  and 
selecting  information  for  all  tasks  in  this 
evaluation. 

14.  Accessing  requested  information  was  diffi¬ 
cult  using  either  access  method  due  to  the 
manner  in  which  the  cursor  moved  across 
the  screen. 

15.  I  would  recommend  another  accessing  method 
rather  than  the  two  methods  provided  in 
this  evaluation. 

16.  I  would  recommend  using  a  light  pen. 

17.  I  would  recommend  using  a  mouse. 

18.  I  would  recommend  using  an  accessing  method 
not  listed. 


4 
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ABP^DdLx  S 

Sample  Interview  Questions 


1.  Did  you  prefer  one  computer  access  method  over  the 
other  one  for  all  steps  in  both  tasks? 

2.  Which  computer  access  method  did  you  prefer?  Why? 

3.  Did  your  access  choice  work  best  in  all  task  steps? 

4.  Which  computer  access  method  worked  best  on  what  tasks? 
Why? 

5.  Which  maintenance  tasks  would  you  recommend  using  the 
joystick  access  method  to  complete? 

6.  Which  maintenance  tasks  would  you  recommend  using  the 
cursor  keys  to  complete? 

7.  Would  another  type  of  computer  access  method  be  more 
practical? 

8.  What  advantages  would  your  suggested  computer  access 
have  over  the  two  access  methods  tested? 

9.  Are  automated  technical  orders  practical  on  the  flight 
line? 

10.  Were  the  steps  in  each  task  representative  of  tasks 
performed  on  the  flight  line? 

11.  Which  tasks  did  not  represent  flight  line  tasks? 

12.  What  recommendation  could  you  offer  that  might  improve 
the  quality  of  this  or  similar  experiments? 

13.  Do  you  think  an  automated  technical  order  has  any 
advantages  over  paper  technical  orders? 

14.  Do  you  have  any  other  recommendations  not  discussed  in 
any  of  the  above  questions? 
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The  following  histograms  present  the  nximber  of 
technicians  who  answered  1  (strongly  agree)  through  5 
(strongly  disagree)  and  the  mean  and  standard  deviation  for 
each  of  the  18  questions  on  the  questionnaire.  Also,  the 
mode  is  listed  along  with  the  percentage  of  technicians  who 
chose  the  mode.  When  there  is  more  than  one  mode,  the 
percentage  shown  is  for  each  separate  value. 


1.  Four-cursor  keys  were  superior  when  accessing  and 
selecting  infomnation  in  the  calculator. 


MEAN  «  2.031  MODE  »  2  (37%)  STD  DEV  =  0.984 


2.  Four-cursor  keys  were  superior  when  accessing  and 
selecting  information  in  all  block  diagrams. 


MEAN  «  2.063  MODE  -  2  (37%)  STD  DEV  -  1.029 
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6.  Four-cursor  keys  is  the  preferred  method  for  accessing 
and  selecting  all  tasks  performed  during  this  evaluation. 


7.  The  thumb  knob  is  superior  when  accessing  and  selecting 
information  in  the  calculator. 


8.  The  thumb  knob  is  superior  when  accessing  and  selecting 
information  in  all  block  diagrams. 


MEAN  >  3.250  MODE  -  3  (34%)  STD  DEV  «  0.935 
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9.  The  thumb  knob  is  superior  when  accessing  and  selecting 
information  in  the  parts  lists. 


1  2  3  4  5  6  7  8  9  10  11  12  13  14  15  16  17  18  19  20  21 


MEAN  >  3.250  MODE  «  2  (31%)  STD  DEV  =  1.061 


10.  The  thumb  knob  is  superior  when  accessing  and  selecting 
information  in  the  menu. 


1  2  3  4  5  6  7  8  9  10  11  12  13  14  15  16  17  18  19  20  21 


MEAN  >  3.219  MODE  -  3  (34%)  STD  DEV  >  0.992 


11.  Fewer  accessing  and  section  errors  were  made  using  the 
thvimb  knob  versus  the  cursor  keys. 


1  2  3  4  5  6  7  8  9  10  11  12  13  14  15  16  17  18  19  20  21 


MEAN  -  3.375  MODE  -  5  (28%)  STD  DEV  >  1.409 
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12.  The  thumb  knob  is  the  preferred  method  for  accessing  and 
selecting  all  tasks  performed  during  this  evaluation. 


13.  Neither  the  cursor  keys  nor  the  thumb  knob  access  method 
worked  best  in  accessing  and  selecting  information  for  all 
tasks  in  this  evaluation. 


14.  Accessing  requested  information  was  difficult  using 
either  access  method  due  to  the  manner  in  which  the  cursor 
moved  across  the  screen. 


MEAN  «  2.938  NODE  «  2  (25%)  STD  DEV  «  1.391 
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15.  I  would  recommend  another  accessing  method  rather  than 
the  two  methods  provided  in  this  evaluation. 
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1. 


Did  you  prefer  one  computer  access  method  over  the 
other  one  for  all  steps  in  both  tasks? 

Only  two  technicians  did  not  have  a  preference. 


2.  Which  computer  access  method  did  you  prefer?  Why? 

Of  the  30  who  did  have  a  preference,  24  chose  the  cursor 
keys. 


3.  Did  your  access  choice  work  best  in  all  task  steps? 

The  cursors  performed  all  tasks  well  for  23  of  24 
respondents.  All  6  technicians  who  liked  the  joystick 
preferred  it  for  every  task. 


12345678  910111213141516171819202122232425 

CURSORS 

YES 

NO 

™ _ 1 _ 1 _ 

JOYSTICK 

YES 

NO 

_ 1 _ 1 _ 
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4.  Which  computer  access  method  worked  best  on  what  tasks? 
Why? 

For  the  cursors:  21  technicians  liked  them  for  every  task, 
while  2  would  not  use  them  to  order  parts  and  one  wouldn't 
employ  them  to  calculate  totals.  For  the  joystick:  3  of  the 
6  preferred  it  for  every  task,  but  4  subjects  disliked  it 
for  the  calculator  and  the  joystick  wasn't  chosen  for  lists 
or  menus  by  one  technician. 


5.  Which  maintenance  tasks  would  you  recommend  using  the 
joystick  access  method  to  complete? 

5  test  subjects  liked  the  joystick  for  all  tasks,  but  12 
didn't  choose  it  at  all.  The  other  15  technicians  some 
functions  including  the  calculator,  scrolling,  part  orders, 
and  menus. 
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6.  Which  compute^!  access  method  would  work  best  on  the 
flight  line' 


23  of  the  30  subjects  with  a  preference  chose  the  cursor 
keys  as  best  for  a  flight  line.  The  other  two  were 
undecided. 


7.  Would  another  type  of  computer  access  method  be  more 
practical?  Why? 

15  test  subjects  did  think  a  different  access  method  would 
be  more  useful,  while  16  were  satisfied  with  the  two  methods 
tested.  One  technician  was  undecided. 


8.  What  advantages  would  your  suggested  computer  access 
method  have  over  the  two  access  methods  tested? 

10  technicians  suggested  a  light  pen,  2  a  mouse  and  one 
desired  a  different  keyboard.  2  others  wanted  an  improved 
joystick.  The  improvements  described  by  the  subjects  are 
listed  below. 
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9.  Are  automated  technical  orders  practical  on  the  flight 
line? 

26  subjects  thought  computerized  TOs  are  practical,  but  one 
didn't  agree.  The  other  five  were  undecided. 


10.  Here  the  steps  in  each  task  representative  of  tasks 
performed  on  the  flight  line? 

27  technicians  thought  that  all  tasks  were  representative  of 
flight  line  tasks,  but  5  did  not. 


11.  Which  tasks  did  not  represent  flight  line  tasks? 

Of  the  5  technicians,  3  of  them  selected  part  ordering  and 
two  picked  the  calculator. 
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12.  What  recommendations  could  you  offer  that  might  improve 
the  quality  of  this  or  similar  experiments? 

Move  the  joystick  to  the  (left)  side  of  the  PMA. 
Presentation  should  allow  diagonal  movement. 

-  Use  an  "ALPHA,"  not  a  "QWERTY"  keyboard. 

Install  a  normal  (or  taller)  joystick. 

Add  a  select  button  as  part  of  joystick. 

Use  tasks  which  match  technicians  career  field. 

Use  CD-ROM  and/or  removable  "hard/ floppy  disks"  to 
update  and  store  information. 

Consider  hazardous  materials  (hydrazine,  skydrol, 
etc)  and  ruggedness  before  approving  the  PMA. 
Incorporate  separate  experience  levels  in 
presentation  system. 

-  Speed-up  response  times. 

Provide  random  access  to  information. 

Change  the  "NEXT"  button  to  "ENTER". 


13.  Do  you  think  an  automated  technical  order  has  any 
advantages  over  paper  technical  orders?  Why? 

31  technicians  thought  automated  TOs  would  improve  flight 
line  operations. 


1234S678  910111213141516171819202122232425262728293031 

YES 

NO 

'm _ 1 _ 1 _ 

REASONS  LISTED  IN  ORDER  BASED  ON  NUMBER  OF  RESPONSES 

Automated  system  easier  to  use  and  operate. 

Easier  to  research  and  find  information  with 
automatic  system. 

New  system  weighs  less,  is  less  bulky  and  faster. 

-  Information  is  more  consolidated  in  IMIS  system. 
Less  storage  space  will  be  required  for  disks  as 
opposed  to  paper  Technical  Orders. 

Less  chances  of  paper  pages  blowing  away. 

Better  survivability  because  paper 
tears/deteriorates . 

-  New  system  will  help  eliminate  mistakes. 

-  New  system  should  cost  less  to  develop. 

-  New  system  will  ensure  information  is  current. 

14.  Do  you  have  any  other  recommendations  not  discussed  in 
any  of  the  above  questions? 

None  mentioned. 
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Air  Force  Specialty  Breakout  of  Technicians  Tested 

31633  -  Apprentice  Instrumentation  Specialists 
31653  ”  Instrumentation  Specialists 

45451  -  Avionics  Guidance  and  Control  Systems  Specialists 

45471  -  Avionics  Guidance  and  Control  Systems  Technician 

45472  -*  Air  crew  Egress  Systems  Technician 

45475  -  Strategic  Electrical  and  Environmental  Systems 
Technician 

45532  -  Apprentice  Communication  and  Navigation  Systems 
Specialist 

45752  -  Airlift  Aircraft  Maintenance  Specialists 
45770  -  Strategic  Aircraft  Maintenance  Technician 
45852  -  Aircraft  Structural  Maintenance  Specialists 
45871  -  Nondestructive  Inspection  Technician 
45873  -  Fabrication  and  Parachute  Supervisor 
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